Quite funny that after a few months offline I find the fun in writing a small article about an investigation which is still about security.
The old applet just signed a hash and I had to find the way to implement this with the new api.
signOld(hash) = signJS(hash, mechanism) where mechanism is a CKM_* (CKM_SHA256_RSA_PKCS, CKM_RSA_PKCS, etc) in PKCS terms.
I am skipping the part where I had to confirm that the same keys are used.
The ebics applet does sign(hash) where hash is a precomputed SHA-256 hash of the data. This led me first to try the CKM_SHA256_RSA_PKCS and CKM_RSA_PKCS mechanisms to replicate the signature in the new api however:
signOld(hash) != signjs(hash, mechanism)
After some investigation (applet decompilation) I confirmed that:
sign(hash) = Signature(hash, "SHA256withRSA")
signOld(hash) != signjs(hash, CKM_SHA256_RSA_PKCS)
The applet used a custom JCE to connect to the hardware token and after more digging I realized
the Digest impl was replaced with a custom digest which did if fact only an Identity. So:
signOld(hash) = Signature(hash, "SHA256withRSA") where SHA256(data) = data*.
Not even now:
signOld(hash) != signjs(hash, CKM_RSA_PKCS)
this is due to the fact (EUREKA) that in fact:
Signature(data, SHA256withRSA) = RSA(OID(SHA256) || SHA256(data))
(see the RFC https://tools.ietf.org/html/rfc3447#section-9.2) which in our case was:
signOld(hash) = RSA(OID(SHA256) || Identity(data)).
This solved the issue as:
signOld(hash) = signjs(OID(SHA256) || hash, CKM_RSA_PKCS)
and OID(SHA256) = “3031300d060960864801650304020105000420”
So instead of signing the first line I should have signed the second in order to replicate the old behaviour
So the interesting result is that:
sign(data, SHA256withRSA) != sign(SHA256(data), NONEwithRSA)