Posts Tagged ‘java’

sign(data, SHA256withRSA) != sign(hash, NONEwithRSA)

Quite funny that after a few months offline I find the fun in writing a small article about an investigation which is still about security.

My task was to find how I could replicate the signature performed with an old applet and a proprietary library which used a hardware token in javascript using a new api provided by a different party.

The old applet just signed a hash and I had to find the way to implement this with the new api.

signOld(hash) = signJS(hash, mechanism) where mechanism is a CKM_* (CKM_SHA256_RSA_PKCS, CKM_RSA_PKCS, etc) in PKCS terms.

I am skipping the part where I had to confirm that the same keys are used.

The ebics applet does sign(hash) where hash is a precomputed SHA-256 hash of the data. This led me first to try the CKM_SHA256_RSA_PKCS and CKM_RSA_PKCS mechanisms to replicate the signature in the new api however:

signOld(hash) != signjs(hash, mechanism)

After some investigation (applet decompilation) I confirmed that:

sign(hash) = Signature(hash, "SHA256withRSA")

but still:

signOld(hash) != signjs(hash, CKM_SHA256_RSA_PKCS)

The applet used a custom JCE to connect to the hardware token and after more digging I realized
the Digest impl was replaced with a custom digest which did if fact only an Identity. So:

signOld(hash) = Signature(hash, "SHA256withRSA") where SHA256(data) = data*. 

Not even now:

signOld(hash) != signjs(hash, CKM_RSA_PKCS)

this is due to the fact (EUREKA) that in fact:

Signature(data, SHA256withRSA) = RSA(OID(SHA256) || SHA256(data)) 

(see the RFC https://tools.ietf.org/html/rfc3447#section-9.2) which in our case was:

signOld(hash) = RSA(OID(SHA256) || Identity(data)).

This solved the issue as:

signOld(hash) = signjs(OID(SHA256) || hash, CKM_RSA_PKCS)

and OID(SHA256) = “3031300d060960864801650304020105000420”

So instead of signing the first line I should have signed the second in order to replicate the old behaviour

fb059955cdc2d5d8f2a3ee78664966576696b7173f14df67031ebe9b63074c84
3031300d060960864801650304020105000420fb059955cdc2d5d8f2a3ee78664966576696b7173f14df67031ebe9b63074c84

So the interesting result is that:

sign(data, SHA256withRSA) != sign(SHA256(data), NONEwithRSA)

Attach payload into detached pkcs#7 signature

If you are doing signature generation via a hardware token (for instance 3Skey) then, for large files it is impractical to send the file to the hardware token. Instead you send a hash (SHA256), get a detached PKCS#7 signature and you need to re-attach the payload in java code. For once this was easier to do with plain JCE code instead of my favorite BouncyCastle provider. However for really large files BC does provide the streaming mechanism required.

Of course the best commands to use to help debug the code bellow are:

Verify pkcs#7 signature

#the -noverify means do not verify the certificate chain, this will only verify the signature not the originating certificate
openssl smime -inform DER -verify -noverify -in signature.p7s

(more…)

An “obvious” improvement

It’s been a long time since I felt such satisfaction debuging something so I decided to write about it.

Let’s assume that you need to store (cache) in memory a large object tree during some operations. In practice this happens because some regulatory constraints so you end up having to parse a very large file and store the resulting object tree. Actually you have a single entry cache. You parse your object, store it in memory for search and processing while the current object tree is used.

public ObjectHandler getObjectHandler(Long id) throws Exception{
	if(cachedObjectHandler != null){
		if(cachedObjectHandler.getId().equals(id)){
			return cachedObjectHandler;
		}
	}
	//else
	cachedObjectHandler = parse(...)
	return cachedObjectHandler;
}

The code above is a simplified way to do it, no? Please note that the parse(…) function creates the object tree by parsing a stream and allocates a new object tree. In my particular case the object tree holded a max of 120k objects (~150Mb) and did some large xml parsing using stax.

So what is wrong with the code above? Take a look at what a single change can do:

public ObjectHandler getObjectHandler(Long id) throws Exception{
	if(cachedObjectHandler != null){
		if(cachedObjectHandler.getId().equals(id)){
			return cachedObjectHandler;
		}
	}
	//else
	cachedObjectHandler = null;
	cachedObjectHandler = parse(...)
	return cachedObjectHandler;
}

Did we just reduced the max needed memory by 2? In the first case since java does right to left assignment first a new object tree is allocated by the parse function and only when done it is assigned to the cachedObjectHandler object allowing for the old object tree to be gc-ed. However with the null assignment it can be gc-ed while the new allocation takes place if memory is needed.

As I said, a small change with a big smile.
 

Java SAML2 + simplesamlphp

The use case is as follows: the java application (SP) must use simplesamlphp as an IdP. I tested 2 libraries, these are the required configs.

SimpleSAMLphp

Please note that the default install from ubuntu (16.04.2) of simplesamlphp (14.0) does not work with the php version installed (php7) because of this bug so I ended installing everything from the tar.gz provided (14.14).

Onelogin

This is the first library I tested. To install it:

  • install maven, requires recent version, does not work with 3.0.5
  • export MAVEN_HOME=/usr/local/java/apache-maven-3.5.0
  • export PATH=$MAVEN_HOME/bin:$PATH
  • git clone https://github.com/onelogin/java-saml
  • cd java-saml
  • mvn package
  • download tomcat 7.0.78
  • install java-saml-toolkit-jspsample as a expanded war in this tomcat
  • tweaked the files: onelogin.saml.properties and the simplesamlphp config until it worked. The key is to use the information from the IdP metadata (http://idp-domain/simplesamlphp/saml2/idp/metadata.php?output=xhtml) and transpose it in the properties file.

Pac4j

This is a more complex library. There is also a demo application for j2e.

To clone, compile and run this demo the sequence is straight forward:

  • git clone https://github.com/pac4j/j2e-pac4j-demo
  • cd j2e-pac4j-demo
  • mvn package
  • mvn jetty:run

The test with https://www.testshib.org works.

To configure if for simpleSAMLphp. Modify DemoConfigFactory.java

final SAML2ClientConfiguration cfg = new SAML2ClientConfiguration("resource:samlKeystore.jks",
 "pac4j-demo-passwd",
 "pac4j-demo-passwd",
 "resource:idp-metadata.xml");
 cfg.setMaximumAuthenticationLifetime(3600);
 cfg.setServiceProviderEntityId("test.pac4j");
 cfg.setServiceProviderMetadataPath(new File("sp-metadata.xml").getAbsolutePath());
 final SAML2Client saml2Client = new SAML2Client(cfg);

The idp-metadata.xml file is the file from: http://idp-domain/simplesamlphp/saml2/idp/metadata.php?output=xhtml wrapped in an additional EntitiesDescriptors element:

<md:EntitiesDescriptor xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> 
 <md:EntityDescriptor entityID="http://idp-domain/simplesamlphp/saml2/idp/metadata.php">

However at this point the application gives a “fatal error”:

org.pac4j.saml.exceptions.SAMLException: Identity provider has no single sign on service available for the selected profileorg.opensaml.saml.saml2.metadata.impl.IDPSSODescriptorImpl@2d6719d3

The error seems to be here https://github.com/pac4j/pac4j/blob/master/pac4j-saml/src/main/java/org/pac4j/saml/context/SAML2MessageContext.java#L104 so I am left with no clue to the problem. The only solution is to change a bit the code to see which is the binding required.

Just cloning the main repository and trying to compile it with maven does not work. The error is:

[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.pac4j:pac4j-couch:[unknown-version]: Could not find artifact org.pac4j:pac4j:pom:2.0.0-RC3-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 5, column 10
@ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR] 
[ERROR] The project org.pac4j:pac4j-couch:[unknown-version] (/phantom/java/pac4j/pac4j-couch/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM for org.pac4j:pac4j-couch:[unknown-version]: Could not find artifact org.pac4j:pac4j:pom:2.0.0-RC3-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 5, column 10 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

The solution is to checkout the 2.0.0 tag:

  • git clone https://github.com/pac4j/pac4j
  • git tag -l
  • git checkout tags/pac4j-2.0.0

At this point I change the code to give the name of the binding:

org.pac4j.saml.exceptions.SAMLException: Identity provider has no single sign on service available for the selected profileurn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST

The solution is to modify the metadata/saml20-idp-hosted.php file and add:

'SingleSignOnServiceBinding' => array('urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect', 'urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST'),
'SingleLogoutServiceBinding' => array('urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect', 'urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST'),

This will generate the

<md:SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"

in the metadata which generated this error.

At this point the SSO works. Of course the entityID for the SP must be configured in metadata/saml20-sp-remote.php

$metadata['diapason.test.pac4j'] = array(
 'AssertionConsumerService' => 'http://localhost:8080/callback?client_name=SAML2Client',

Read fast or die

I have spend a lot of time today trying to find and fix an issue which ended up to be a fun discovery at the end.

The following java error occurred when loading a pdf file from an url stream:

java.io.IOException: missing CR
at sun.net.www.http.ChunkedInputStream.processRaw(ChunkedInputStream.java:405)
at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:572)
at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609)
at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3066)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3060)

This looked like a java lib error since java version was a bit old so the first idea was to replace the code with some apache httpClient based code to load the URL. This generated the following error, very similar.

java.io.IOException: CRLF expected at end of chunk: 121/79
at org.apache.commons.httpclient.ChunkedInputStream.readCRLF(Unknown Source)
at org.apache.commons.httpclient.ChunkedInputStream.nextChunk(Unknown Source)
at org.apache.commons.httpclient.ChunkedInputStream.read(Unknown Source)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.commons.httpclient.AutoCloseInputStream.read(Unknown Source)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.commons.httpclient.AutoCloseInputStream.read(Unknown Source)

Since this was a windows machine and the requests passed via localhost another try was to use another interface. The result was similar.

After some search I found a nice tool: http://www.netresec.com/?page=RawCap which does not requires any install and can be used even on localhost to generate a pcap compatible file which can then be inspected in wireshark.

The result was strange. Opening the capture in wireshark on my machine showed: [7540 bytes missing in capture file] in the tcp stream. This corresponded to a lot of packets: [TCP ZeroWindow], [TCP ZeroWindowProbe].

Selection_655

Since this was a VMWare install and I previously had some trouble with vmware software switches I assumed this was related to the network card config however it also happened on localhost.

After some more investigation I realized this was only happening when several request where made in parallel. I confirmed this by looking in the code.

The code contained a worker pool. Each worker/thread constructed the url the used then returned an inputStream.

DataSource source = new URLDataSource(reportUrl);
return source.getInputStream();

However all the results were handled in sequence. As such while one url inputStream was read, the server continued to send data on all others request but this data was not read on the client side fast enough. As a result the tcp window got indeed to 0 and the strange error occurred.

Of course the solution was to fully read the data in each worker.

The difficult to find bug

After upgrading from 3.4 to 3.6 my JGroups code stopped working. On a 2 nodes setup when the second node tried to join I got the following errors:

2015-04-22 21:38:12,510 INFO  [ViewHandler,monalisa,tux-7762|fr.mcc.test.TestJGroups] Detected a change in cluster members: [tux-7762, tux-9846]
2015-04-22 21:38:14,460 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 1
2015-04-22 21:38:16,463 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 2
2015-04-22 21:38:18,466 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 3
2015-04-22 21:38:20,468 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 4
2015-04-22 21:38:22,471 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 5
2015-04-22 21:38:24,473 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 6
2015-04-22 21:38:26,475 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 7
2015-04-22 21:38:28,479 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 8
2015-04-22 21:38:30,482 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 9
2015-04-22 21:38:32,485 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: JOIN(tux-9846) sent to tux-7762 timed out (after 2000 ms), on try 10
2015-04-22 21:38:32,485 WARN  [main|org.jgroups.protocols.pbcast.GMS] tux-9846: too many JOIN attempts (10): becoming singleton
2015-04-22 21:38:32,493 WARN  [Incoming-2,monalisa,tux-9846|org.jgroups.protocols.pbcast.NAKACK2] JGRP000011: tux-9846: dropped message 1 from non-member tux-7762 (view=[tux-9846|0] (1) [tux-9846])

It took some time but finally I managed to write a “simple” (quick and ugly) test to replicate the problem.
(more…)

ojdbc14.jar to ojdbc6.jar migration

Migrating from ojdbc14.jar to ojdbc6.jar is not, as one might think, completely seamless. Here are at least 3 points which required code change:

Changes in class hierarchy

error: OracleTypes is not public in oracle.jdbc.driver; cannot be accessed from outside package
 [javac] private static Integer OracleRefCursorType = new Integer(oracle.jdbc.driver.OracleTypes.CURSOR);

(more…)

Replicated EhCache, the uneasy road

At first replicating EhCache seems a very easy task, just need to configure ehcache.xml with RMI and you are ready. Is it so?

(more…)

Database locking

This is a very simple example demonstrating a method to achieve a reliable lock in a cluster when the cluster shares a database connection. This example uses an Oracle database which has the following table/row:

CREATE TABLE locked(shortname varchar2(100) PRIMARY KEY, info varchar2(100));
INSERT INTO locked VALUES('TestLock', '');
commit;

(more…)

Let’s decrypt

AES encrypt in java and decrypt in java, flex, python, C#.

Encrypt: java

public static void encrypt(InputStream is, OutputStream out, String secret) throws Exception {
	SecretKey secretKey = new SecretKeySpec(Hex.decodeHex(secret.toCharArray()), "AES");
        Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
        cipher.init(Cipher.ENCRYPT_MODE, secretKey);
        out.write(cipher.getParameters().getParameterSpec(IvParameterSpec.class).getIV());
        CipherOutputStream cipherOutputStream = new CipherOutputStream(out, cipher);
        int bufLength = KEY_LENGTH/8 * 100;
        byte buf[] = new byte[bufLength];
        int bytesRead = 0;
	while((bytesRead = is.read(buf)) != -1 ) {
		cipherOutputStream.write(buf, 0, bytesRead);
	}
	cipherOutputStream.close();
}

(more…)