Thursday, November 14, 2013

install jdk7 in ubuntu

To install jdk7 in ubuntu follow next step
sudo apt-get install python-software-properties
if dont install this error is show "Ubuntu: add-apt-repository: command not found" when execute next step
 sudo add-apt-repository ppa:webupd8team/java
 sudo apt-get update
 sudo apt-get install oracle-java7-installer

Monday, September 2, 2013

make a mirror dbpedia in 5 steps

to make a mirror, you follow 5 simples steps. you must install virtuoso database. and change virtuoso.ini to read files
DirsAllowed              = ., /path/files/script 
1º download files from dbpedia with command
wget -r -np -nd -nc -A'*.nt.bz2' http://downloads.dbpedia.org/3.8/en/
2º decompress files
for i in `ls ./download/*.bz2` ; do
  bzcat $i | gzip --fast > ${i%.bz2}.gz && mv ./download/*.gz ./decompress/ && rm $i ;
done &
3º deletes triplets more long 1024 chars.
cd decompress;
for i in *.gz  ; do
  echo -n "cleaning $i..."
  zcat $i | grep -v -E '^<.+> <.+> <.{1025,}> \.$'  > $i.clean
  mv *clean ../fixed/
  echo "done."
done
4º split files in small files.
cd fixed
for i in *clean ; do

  split_size=$(stat -c%s "$i")

  echo "Size = $split_size"

  echo -n "slipting  $i..."

  if [ $split_size -gt 5000000 ]; then
        echo "File is large."

        echo "Performing split on file $i"

        split -a 10 -l 50000 $i "split/$i" ;
  else
        mv $i split/ ;
   fi

done;
5º execute insert command in virtuoso database
echo "creating load statement"

cd  slipt

for file in `ls *`
do
        load_target="/home/user/split/$file"
        load_query="EXEC=TTLP_MT(file_to_string_output('$load_target'), '', 'http://dbpedia.org', 255);commit WORK;"
        isql-vt 1111  dba dba  "$load_query"
     #  echo "$virt_isql" "$virt_port" "$virt_userName" "$virt_passWord" "$load_query"
 if [ $?  -ne 1 ]; then
  #echo "execution ok. move data"
  ls  split/ | wc -l
  mv "$load_target"  process-files/
  echo "procesando $file"
 else
   echo "error to process file $file"
   exit;
 fi
done;
echo "done"

Tuesday, August 27, 2013

Eclipse + maven: error compilation suddenly

Sometimes, you can see this error in eclipse.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] Unable to locate the Javac Compiler in:
  C:\Program Files (x86)\Java\jre7\..\lib\tools.jar
Please ensure you are using JDK 1.4 or above and
not a JRE (the com.sun.tools.javac.Main class is required).
In most cases you can change the location of your Java
installation by setting the JAVA_HOME environment variable.
[INFO] 1 error
This error happen when you compile eclipse and maven with jre. You must change to JDK compile in Eclise.

Wednesday, August 21, 2013

read a resource inisde a jar

In both examples the file has locate in root of jar file
this.getClass().getClassLoader().getResourceAsStream("file.xml"))
this.getClass().getResourceAsStream("/file.xml"))



this.getClass().getClassLoader().getResource("file.xml").getFile())

creacion de un cluster con Lily en cloudera

Encontre hace poco un proyecto muy interesante http://www.lilyproject.org/lily/index.html
Para instalarlo en clustes hay que seguir la guía pero te encontrarás con algún problema.
http://lilyproject.org/books/lily_book--2_0--20130124-120744/publications/html-chunked/output/dsy432-lily.html#dsy432-lily Recientemente he creado un cluster con 3 equipos en rack de Lily. Primero instalo cloudera en cada uno de los 3 equipos pero me pedia actualizar a la version de 12 de ubuntu.
Una vez actualizado me han ido dando distintos errores:

  • sncronizacion de los relojes.
    La sincronización de los relojes de los equipos, tiene que dar una diferencia menor de 3 segundos. Con un servicio de NTP en ubuntu, esto se ha corregido. El resto de los equipos se conecta a el. Si no se sincroniza correctamente verifica la fecha hora de tu equipo. Verifica que si tienes un dominio, estén todos en dominio, ya que si uno de ellos no lo está, fallará constantemente y no usará el servicio de NTP.
  • hdfs al arrancar da un error.
    se soluciona borrando la carpeta a mano.
  • mapreduce-traker da un error de versiones distinta, se solucion actualizan todo el sistema.
  • HBase no arranca bien,
    HRegionServer Failed deleting my ephemeral node org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired esto se debe a un error de sincronización.
  • Para poder lanzar Lily en Cloudera se debe tener autenticacion con Kerberos si te da un error parecido a este "Authentication failed. Cannot get kdc for realm" es que no tienes kerberos instalados. http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Security-Guide/cdh4sg_topic_11_1.html tutorial de kerberos https://we.riseup.net/debian/kerberos-for-birds-not-dummies primero se debe instalar kerberos install kerbero atp-get install krb5-admin-server

    change version java runtime in tomcat7

    Tomcat dont execute default version java in linux machine. to view what version java use proccess tomcat type this
    #ps -ef | grep tomcat
    
    tomcat7  13692     1  1 10:57 ?        00:00:07 /usr/lib/jvm/default-java/bin/java -Djava.util.logging.config.file=/var/lib/tomcat7/conf/logging.properties -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/share/tomcat7/endorsed -classpath /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat7 -Dcatalina.home=/usr/share/tomcat7 -Djava.io.tmpdir=/tmp/tomcat7-tomcat7-tmp org.apache.catalina.startup.Bootstrap start
    
    To view default java version
    #java -version 
    
    java version "1.7.0_21"
    Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
    Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)
    
    and execute default java
    #/usr/lib/jvm/default-java/bin/java -version
    
    
    java version "1.6.0_27"
    OpenJDK Runtime Environment (IcedTea6 1.12.5) (6b27-1.12.5-0ubuntu0.12.04.1)
    OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
    
    #update-java-alternatives  -l
    
    j2sdk1.6-oracle 315 /usr/lib/jvm/j2sdk1.6-oracle
    java-1.6.0-openjdk-amd64 1061 /usr/lib/jvm/java-1.6.0-openjdk-amd64
    java-1.7.0-openjdk-amd64 1051 /usr/lib/jvm/java-1.7.0-openjdk-amd64
    java-7-oracle 1062 /usr/lib/jvm/java-7-oracle
    
    show soft link to java 1.6
    #/usr/lib/jvm/ ls -la 
    default-java -> java-1.6.0-openjdk-amd64
    
    remove soft link to 1.6 and create a new link 1.7
    # unlink default-java
    # ln -f  -s java-7-oracle /usr/lib/jvm/default-java
    
    show new link to 1.7
    #/usr/lib/jvm/ ls -la 
    default-java -> java-7-oracle
    

    Monday, July 15, 2013

    unix script file exists

    I have a lists of files. Inside each file I have a list of files. I have to test that this files exist in directory /tmp I create this scprit to test pattern list of files dbpediaa? #pattern=`ls -la | awk '{print $9}' | grep -vE "\."`
    
    for t in dbpediaa?
    do
     echo "existe fichero $t ? "
    
     ficheros=`grep -i .nt.gz  $t | awk -F "'"  '{ print $4 }'`
    
     
     for fichero in $ficheros
     do
       if [ -f /tmp/$fichero  ]; then
         echo "."
        else
         echo "no existe $fichero"
       fi
     done;
    done;
    
    

    Thursday, June 27, 2013

    publish with maven in tomcat7

    There're a bug in plugin
       org.apache.tomcat.maven
       tomcat7-maven-plugin
       2.1
    
    dont read the tag configuration you must put each var in section properties
          
      http://localhost:port/manager/text
      TomcatServer
      ${project.artifactId}
      true
      /${project.build.finalName}
     
    
    and execute goal mvn tomcat:redeploy this work¡

    Wednesday, June 5, 2013

    integration-test with web server

    I want created a integration-test with web server tomcat to launch integration test execute
    mvn integration-test -Ptomcat 
    
    to generate war file
    mvn package
    
    In a hand i have web server tomcat: to launch integration-test i have a this file pom.xml to start and stop tomcat web service
    
       tomcat
       
     
          org.codehaus.mojo
          tomcat-maven-plugin
          1.1
          
           true
           true
           /${project.build.finalName}
           ${project.build.directory}/${project.build.finalName}.war
           9876
          
          
           
            start-tomcat
            pre-integration-test
            
             run-war
            
           
           
            stop-tomcat
            post-integration-test
            
             shutdown
            
           
          
         
    
    and the other hand i have the integration test
    
       
        maven-surefire-plugin
        
         true
        
       
       
        maven-failsafe-plugin
        2.7.2
        
         
          **/*Test.java
         
        
        
         
          integration-test
          integration-test
          
           integration-test
          
         
         
          verify
          verify
          
           verify
          
         
        
       
      
    
    But I have a problem to add resources files. you need add this line in your pom.xml file
    
         
          src/test/resources
          
           app.properties
          
         
        
    
    and add this line in pom.xml file to erase previous resources files generate with execute integration test
    
          org.apache.maven.plugins
          maven-clean-plugin
          2.5
          
           
            clean
            generate-resources
            
             clean
            
           
          
    
    

    Thursday, May 30, 2013

    DSE cassandra: dangerous command apt-get upgrade

    there 're an error when upgrade version 1.1.11. to version 1.2.4 in cassandra
    si por algun casual se te ocurre actualizar el sistema de cassandra de la version 1.1.11 a la version 1.2.4, te encontraras con el siguiente error
    INFO [main] 2012-04-17 12:12:02,065 AbstractCassandraDaemon.java (line 105) Logging initialized
    INFO [main] 2012-04-17 12:12:02,123 DatabaseDescriptor.java (line 116) Loading settings from file:/home/vensocia/datastax/resources/cassandra/conf/cassandra.yaml
    ERROR [main] 2012-04-17 12:12:02,216 DatabaseDescriptor.java (line 448) Fatal configuration error error
    while parsing a block mapping
    in "", line 10, column 1:
    cluster_name: 'Test Cluster'
    ^
    expected , but found BlockMappingStart
    
    dont modify the file cassandra.yaml
    No te vuelvas loco modificando el fichero cassandra.yaml. Tienes dos opciones sigue las instrucciones de http://www.datastax.com/docs/datastax_enterprise2.0/upgrading_dse o revierte los cambios. execute command to show cassandra version
    ejecuta el comando para ver que versión de cassandra tienes
    dpkg -s cassandra
    
    and make a rollback
    y vuelve a dejarlo todo como estaba con la instruccion
    sudo apt-get install dsc1.1 
    

    Wednesday, March 20, 2013

    Solandra install

    download git and gittortoise download solandra form https://github.com/tjake/Solandra.git with cloneRepository download and install maven3 http://maven.apache.org/download.cgi and configure settings repository and proxy prerequisites jdk, mvn configure java_home 1.6 jdk, (note that java7 don`t work) and mvn_home download and put in directory Solandra\reuters-demo app curl http://curl.haxx.se/latest.cgi?curl=win64-nossl modify bach file Solandra\reuters-demo

    Cassandra-cli command

    Select database or keyspace [default@unknown] USE demo; List all user of database demo with limit 100 , if change limit bigint the cassandra-cli colapse. [default@demo] LIST users; http://www.datastax.com/docs/0.8/dml/using_cli list all keyspace [default@unknown] show keyspace;

    Thursday, January 17, 2013

    Make inference with pellet in Virtuoso

    
      set = new VirtGraph(GraphIri, "jdbc:virtuoso://" + uriServer + ":"
        + portJDBC + "", db_user, db_pass);
      vd = new VirtDataSource("jdbc:virtuoso://" + uriServer + ":" + portJDBC
        + "", db_user, db_pass);
      System.out.println("Start inference!");
      Reasoner reasoner = PelletReasonerFactory.theInstance().create();
      reasoner = reasoner.bindSchema(set);
      infModel = ModelFactory.createInfModel(reasoner, modelVirtuoso);
      System.out.println("Stop inference!");
      Model m2 = vd.getNamedModel(GraphIri);
      Model inferido = infModel.difference(m2);
      m2.add(inferido);
    

    Tuesday, January 8, 2013

    good examples of Dependence Injections

    The first link is a simple tutorial Start
    The second is a interesting aplication of AOP Interceptions and the last are Scope Scopes