Posts Tagged ‘java’

Read fast or die

I have spend a lot of time today trying to find and fix an issue which ended up to be a fun discovery at the end.

The following java error occurred when loading a pdf file from an url stream:

java.io.IOException: missing CR
at sun.net.www.http.ChunkedInputStream.processRaw(ChunkedInputStream.java:405)
at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:572)
at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609)
at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3066)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3060)

This looked like a java lib error since java version was a bit old so the first idea was to replace the code with some apache httpClient based code to load the URL. This generated the following error, very similar.

java.io.IOException: CRLF expected at end of chunk: 121/79
at org.apache.commons.httpclient.ChunkedInputStream.readCRLF(Unknown Source)
at org.apache.commons.httpclient.ChunkedInputStream.nextChunk(Unknown Source)
at org.apache.commons.httpclient.ChunkedInputStream.read(Unknown Source)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.commons.httpclient.AutoCloseInputStream.read(Unknown Source)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.commons.httpclient.AutoCloseInputStream.read(Unknown Source)

Since this was a windows machine and the requests passed via localhost another try was to use another interface. The result was similar.

After some search I found a nice tool: http://www.netresec.com/?page=RawCap which does not requires any install and can be used even on localhost to generate a pcap compatible file which can then be inspected in wireshark.

The result was strange. Opening the capture in wireshark on my machine showed: [7540 bytes missing in capture file] in the tcp stream. This corresponded to a lot of packets: [TCP ZeroWindow], [TCP ZeroWindowProbe].

Selection_655

Since this was a VMWare install and I previously had some trouble with vmware software switches I assumed this was related to the network card config however it also happened on localhost.

After some more investigation I realized this was only happening when several request where made in parallel. I confirmed this by looking in the code.

The code contained a worker pool. Each worker/thread constructed the url the used then returned an inputStream.

DataSource source = new URLDataSource(reportUrl);
return source.getInputStream();

However all the results were handled in sequence. As such while one url inputStream was read, the server continued to send data on all others request but this data was not read on the client side fast enough. As a result the tcp window got indeed to 0 and the strange error occurred.

Of course the solution was to fully read the data in each worker.

The difficult to find bug

After upgrading from 3.4 to 3.6 my JGroups code stopped working. On a 2 nodes setup when the second node tried to join I got the following errors:

2015-04-22 21:38:12,510 INFO  [ViewHandler,monalisa,tux-7762|fr.mcc.test.TestJGroups] Detected a change in cluster members: [tux-7762, tux-9846]
2015-04-22 21:38:14,460 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 1
2015-04-22 21:38:16,463 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 2
2015-04-22 21:38:18,466 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 3
2015-04-22 21:38:20,468 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 4
2015-04-22 21:38:22,471 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 5
2015-04-22 21:38:24,473 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 6
2015-04-22 21:38:26,475 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 7
2015-04-22 21:38:28,479 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 8
2015-04-22 21:38:30,482 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 9
2015-04-22 21:38:32,485 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 10
2015-04-22 21:38:32,485 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: too many JOIN attempts (10): becoming singleton
2015-04-22 21:38:32,493 WARN  [Incoming-2,monalisa,tux-9846|org.jgroups.protocols.pbcast.NAKACK2] JGRP000011: tux-9846: dropped message 1 from non-member tux-7762 (view=[tux-9846|0] (1) [tux-9846])

It took some time but finally I managed to write a “simple” (quick and ugly) test to replicate the problem.
(more…)

ojdbc14.jar to ojdbc6.jar migration

Migrating from ojdbc14.jar to ojdbc6.jar is not, as one might think, completely seamless. Here are at least 3 points which required code change:

Changes in class hierarchy

error: OracleTypes is not public in oracle.jdbc.driver; cannot be accessed from outside package
 [javac] private static Integer OracleRefCursorType = new Integer(oracle.jdbc.driver.OracleTypes.CURSOR);

(more…)

Replicated EhCache, the uneasy road

At first replicating EhCache seems a very easy task, just need to configure ehcache.xml with RMI and you are ready. Is it so?

(more…)

Database locking

This is a very simple example demonstrating a method to achieve a reliable lock in a cluster when the cluster shares a database connection. This example uses an Oracle database which has the following table/row:

CREATE TABLE locked(shortname varchar2(100) PRIMARY KEY, info varchar2(100));
INSERT INTO locked VALUES('TestLock', '');
commit;

(more…)

Let’s decrypt

AES encrypt in java and decrypt in java, flex, python, C#.

Encrypt: java

public static void encrypt(InputStream is, OutputStream out, String secret) throws Exception {
	SecretKey secretKey = new SecretKeySpec(Hex.decodeHex(secret.toCharArray()), "AES");
        Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
        cipher.init(Cipher.ENCRYPT_MODE, secretKey);
        out.write(cipher.getParameters().getParameterSpec(IvParameterSpec.class).getIV());
        CipherOutputStream cipherOutputStream = new CipherOutputStream(out, cipher);
        int bufLength = KEY_LENGTH/8 * 100;
        byte buf[] = new byte[bufLength];
        int bytesRead = 0;
	while((bytesRead = is.read(buf)) != -1 ) {
		cipherOutputStream.write(buf, 0, bytesRead);
	}
	cipherOutputStream.close();
}

(more…)

JBoss migration – Quartz

This is a continuation of the previous article regarding some migration points (1, 2) from JBoss 4.2.2-GA to JBoss 7.1.1 and, presumably, Tomcat 7.

3. Quartz

Quartz migration has been the simplest of all, by far.
(more…)

JBoss migration – the HAR archive

This is a continuation of the previous article regarding some migration points from JBoss 4.2.2-GA to JBoss 7.1.1 and, presumably, Tomcat 7.

2. The HAR archive

The HAR archive was a nice mechanism which allowed hibernate integration. A ${name}.har file was created, containing all the mappings (*.hbm.xml) and data classes (*.class), allong with a hibernate-service.xml (later renamed to service-hibernate.xml in JBoss 5). This took care of creating the SessionFactory and making it accessible through JNDI. Creating a session in code become:

InitialContext ctx = new InitialContext();
SessionFactory factory = (SessionFactory) ctx.lookup("XName");
return factory.openSession();

(more…)

JBoss migration – the data source

I have spent a lot of time lately trying to create a migration plan for an application currently running on JBoss 4.2.2. Since this application development started a few migration attempts to newer versions of JBoss have been done (see for 5.1) but as it seems each version has different style configuration files and this application is expected to have a long lifetime the work seems a bit futile so in parallel of
migrating to JBoss-7.1.1 I’ve tried to migrate to Tomcat since our use of EJB’s is limited and other MBeans can be refactored in simpler ways. Unlike JBoss, Tomcat configuration seems to be eternal and by all means simpler.
(more…)

To migrate or not to migrate

This is not a guide, nor intended to help, it’s a steam valve for my efforts to migrate an application to jboss 7 as each exception can take minutes or hours to solve without altering the original code.

Caused by: org.jboss.jca.common.metadata.ParserException: IJ010061: Unexpected element: local-tx-datasource
        at org.jboss.jca.common.metadata.ds.DsParser.parseDataSources(DsParser.java:183)
        at org.jboss.jca.common.metadata.ds.DsParser.parse(DsParser.java:119)
        at org.jboss.jca.common.metadata.ds.DsParser.parse(DsParser.java:82)
        at org.jboss.as.connector.deployers.processors.DsXmlDeploymentParsingProcessor.deploy(DsXmlDeploymentParsingProcessor.java:80)

        at org.jboss.as.ee.metadata.MethodAnnotationAggregator.runtimeAnnotation
Information(MethodAnnotationAggregator.java:58)
        at org.jboss.as.ee.component.deployers.InterceptorAnnotationProcessor.ha
ndleAnnotations(InterceptorAnnotationProcessor.java:85)
        at org.jboss.as.ee.component.deployers.InterceptorAnnotationProcessor.pr
ocessComponentConfig(InterceptorAnnotationProcessor.java:70)
        at org.jboss.as.ee.component.deployers.InterceptorAnnotationProcessor.de
ploy(InterceptorAnnotationProcessor.java:55)
        at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:113) [jboss-as-server-7.1.1.Final.jar:7.1.1.Final]
        ... 5 more
Caused by: java.lang.NoClassDefFoundError: org/hibernate/HibernateException
        at java.lang.Class.getDeclaredFields0(Native Method) [rt.jar:1.6.0_23]
        at java.lang.Class.privateGetDeclaredFields(Class.java:2291) [rt.jar:1.6.0_23]
        at java.lang.Class.getDeclaredFields(Class.java:1743) [rt.jar:1.6.0_23]
        at org.jboss.as.server.deployment.reflect.ClassReflectionIndex.(ClassReflectionIndex.java:57) [jboss-as-server-7.1.1.Final.jar:7.1.1.Final]
        at org.jboss.as.server.deployment.reflect.DeploymentReflectionIndex.getClassIndex(DeploymentReflectionIndex.java:66) [jboss-as-server-7.1.1.Final.jar:7.1.1.Final]

        at org.apache.catalina.core.StandardContext.contextListenerStart(StandardContext.java:3392) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:3850) [jbossweb-7.0.13.Final.jar:]
        at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_23]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_23]
        at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_23]
Caused by: java.lang.ClassNotFoundException: org.jboss.as.jmx.PluggableMBeanServerBuilder from [Module "deployment.diapason.ear.diapason-web.war:main" from Service Module Loader]

(MSC service thread 1-6) Exception sending context initialize
d event to listener instance of class org.jbpm.web.JobExecutorLauncher: org.jbpm
.JbpmException: couldn't parse jbpm configuration from resource 'jbpm.cfg.xml'
        at org.jbpm.JbpmConfiguration.getInstance(JbpmConfiguration.java:292) [j
bpm-jpdl.jar:3.2.3 (date:18-Jun-2008 00:51)]
        at org.jbpm.web.JobExecutorLauncher.contextInitialized(JobExecutorLaunch
er.java:55) [jbpm-jpdl.jar:3.2.3 (date:18-Jun-2008 00:51)]
        at org.apache.catalina.core.StandardContext.contextListenerStart(StandardContext.java:3392) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:3850) [jbossweb-7.0.13.Final.jar:]
        at org.jboss.as.web.deployment.WebDeploymentService.start(WebDeploymentService.java:90) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)

host].[/dix]] (MSC service thread 1-4) StandardWrapper.Throwable: java.lang.NoClassDefFoundError: org/mozilla/javascript/WrappedException
        at java.lang.Class.forName0(Native Method) [rt.jar:1.6.0_23]
        at java.lang.Class.forName(Class.java:169) [rt.jar:1.6.0_23]

ava.lang.IllegalAccessError: tried to access class 
org.hibernate.cfg.Configuration$1 from class org.hibernate.cfg.Configuration
08:21:17,924 ERROR [stderr] (JbpmJobExecutor:127.0.1.1:1)       at org.hibernate
.cfg.Configuration.buildMapping(Configuration.java:2923)
08:21:17,925 ERROR [stderr] (JbpmJobExecutor:127.0.1.1:1)       at org.hibernate
.cfg.Configuration.(Configuration.java:269)
08:21:17,925 ERROR [stderr] (JbpmJobExecutor:127.0.1.1:1)       at org.hibernate
.cfg.Configuration.(Configuration.java:302)
08:21:17,925 ERROR [stderr] (JbpmJobExecutor:127.0.1.1:1)       at org.jbpm.db.h
ibernate.HibernateHelper.createConfiguration(HibernateHelper.java:74)

from Service Module Loader: java.lang.LinkageError:

Caused by: java.lang.NoClassDefFoundError: org/jboss/as/clustering/infinispan/subsystem/CacheConfigurationService
        at org.jboss.as.jpa.hibernate3.HibernatePersistenceProviderAdaptor.addProviderDependencies(HibernatePersistenceProviderAdaptor.java:104)
        at org.jboss.as.jpa.processor.PersistenceUnitDeploymentProcessor.deployPersistenceUnit(PersistenceUnitDeploymentProcessor.java:345)
        at org.jboss.as.jpa.processor.PersistenceUnitDeploymentProcessor.addPuService(PersistenceUnitDeploymentProcessor.java:258)
        at org.jboss.as.jpa.processor.PersistenceUnitDeploymentProcessor.handleEarDeployment(PersistenceUnitDeploymentProcessor.java:216)
        at org.jboss.as.jpa.processor.PersistenceUnitDeploymentProcessor.deploy(PersistenceUnitDeploymentProcessor.java:119)

And so on…