Category Archives: Uncategorized

Using PyCrypto with Spring Crypto/Spring Security Default Encoders

Spring Crypto Encryptor Details

The Spring Crypto module is amazing. Secure defaults using standard interfaces really make it a pleasure to use, and with it being full tested and vetted, it also gives a develoepr the warm fuzzy feelings of nice Java security implentation.

The standard interface is through the org.springframework.security.crypto.encrypt.Encryptors class, defined (in version 3.2.0 here (the class API to which this blog post was written in July 2016).

Spring Crypto uses AES256 encryption behind the scenes in its out of the box class setup; it’s as easy as:

https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=spring-crypto-example-setup.java

In the above code snippet, the password variable is a passcode, with the salt variable used to create the AES key. It uses 256 bit encryption with the standard calls (depending on your JRE/Java Cryptography Extension (JCE) Setup, make sure to download the appropriate extension). The default AESBytesEncryptor setup generates a 256 bit key (32 bytes) from a salted iteration done 1024 times. Another thing to note here, is that the salt variable is decoded from a 16 character Hex string and the default Java implementation is “AES/CBC/PKCS5Padding“. The PKCS#5 padding is defined in its own RFC, and it is how to deal with byte arrays that are not an integer multiple of a given byte length (16).

Here is the relevant code in the AESBytesEncryptor class:

https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=AESBytesEncryptor.java

With the default text encoder, it uses a random 16 byte Initialization Vector (IV), prepended to the encrypted text hex output String. What this means is that the same string encoded multiple times will result in different outputs.

https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=encryption-usage.java

Produces the following output:
https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=encryption-usage-output.txt

As you can see, the same string “foo” encrypted multiple times results in different encrypted outputs. This is because the initialization vector IV. In the AESBytesEncryptor class, the encrypt method has the following lines of code:

https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=AESBytesEncryptor-2.java

The perceptive reader will also note the ternary switch statement with the byte-concatination; if the NULL_IV_GENERATOR is used, then the encrypted text is returned directly. The concept in the Spring Crypto world is ‘queryable encryption‘. The Encryptor class has the ‘queryableText‘ encryptor, which uses the null IV concept. The encrypted text is ‘queryable’, because it returns the same string output each time for the same input string.

https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=queryable-encryption-usage.java
Produces the following identical hex strings:

https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=queryable-encryption-usage-output.txt

Two little more snippets of code to fully understand how the spring-crypto module details work. The input String is encoded with UTF-8 before being encrypted,  and then a hex encoding/decoding is applied.

Here is the encoding call from the HexEncodingTextEncryptor class:
https://gist.github.com/tmarthal/6dbec6cc358ab78ea090747499c57981.js?file=HexEncodingTextEncryptor.java

With the decryption decoding in a similar manner. That is the overview of the Spring Security/Spring Crypto implementation details.

PyCrypto Implementation Details

Now that we understand the encodings, paddings and encryption algorithms presented in the spring-crypto library, the super powerful and fully configurable pycrypto/pycryptodome library can easily be setup to encrypt/decrypt the same hex strings. The below snippets use python 3.5 and pycryptodome as the PyCrypto implementation.

Here is the linked implementation in the custom AES256 implementation AesCrypt256 python class (source too long to embed).

With the random initialization vector example usage specified:
https://gist.github.com/tmarthal/cf5a610c5c5ab1e661a6351c96200706.js?file=sample-usage.py

And the example output:
https://gist.github.com/tmarthal/cf5a610c5c5ab1e661a6351c96200706.js?file=sample-usage-output.txt

And to run the sample code with the null/16-byte \x00 vector:
https://gist.github.com/tmarthal/cf5a610c5c5ab1e661a6351c96200706.js?file=sample-usage-no-iv.py
https://gist.github.com/tmarthal/cf5a610c5c5ab1e661a6351c96200706.js?file=sample-usage-no-iv-output.txt

Combining outputs:

The hex string that we get from the spring-crypto library can both be decrypted with the python snippet. See the gist for embedded output, with each of the random IV encrypted values as well as the no-IV encrypted values for the string ‘foo’:

https://gist.github.com/tmarthal/cf5a610c5c5ab1e661a6351c96200706.js?file=sample-input-decryption.py

Good luck!

LA Public Library Overlays, Take 1 redux

I wanted to start a project that would try increase the use of the local LA County library system. To do so, I thought it would be awesome to get more people to stop buying books online and instead checkout books from their local library.

I still need to work out some kinks, but I created a github repo that will overlay any links (with ISBN-10 urls — a small subset) to purchase a book from Amazon to re-direct to the LA County Library website search.

For instance, say there are links in the page like the following:

City of Dragons by Robin Hobb
O’Reilly’s Information Architecture for the Web (This was highlighted by the librarian when I was asking about API access :P)

Right now there is nothing special about the links. You can hover over them with your mouse to see that they point to Amazon.

To see the code in action

  1. [If using chrome: Drag this link onto the current tab][With Safari, drag it onto your Bookmarks bar, then click on the bookmarked link][Other browsers: TBD]:
    COLA Public Amazon Highlight

    Assuming that the github js link works, if you click on the link, and then re-hover over the previous links you will see API calls to the LA County website.

  2. Hover your cursor over the above Amazon links

You should see something like:
Which link will direct you to the Los Angeles Colapublic Library page to reserve the book.


Login, place the book on hold and the library system will email you when the book is available to pickup at your local library.

To create a bookmarklet and get this functionality across the web, you can add the above javascript: link to your browser bookmarks. Bookmarks with javascript code are called bookmarklets then when you want the overlay functionality, simply select the bookmark(let)

Technically, there is nothing special in this iteration, but just a general idea.

Note:This blog was originally published on tumblr, but there where problems running 3rd party javascript to create in-page divs.

Blog Code Formatting

Note (2012): This post was about a previous webdesign, and no longer applicable.

I finally added code formatting to my blog. I used a completely ‘client side’ javascript library called syntaxhighlighter, hosted on Google Code. Note that I’m currently linking javascript directly from their web facing SVN repository, so I’m not sure how stable it is (but hosting it on googlepages didn’t seem to work (they disallow js linking I believe)).

Although it is a technology blog, I thought I posted more code about stuff. I only seem to have two posts where I actually put up any code (But maybe that’s because it was horribly formatted?), and they now both have pretty Java and XML formatting:

Maybe one of these days, I’ll find use for a blog again. The whole concept of blogging seems so dead.

My Yahoo! Answers Science Post

I am my dad’s personal Yahoo! answers person when it comes to things science and physics. So when I received a question on the dual nature of light and matter, I wrote him back a nice long (LFP!) email in answer. I thought I’d share.

Question:

Sound travels in waves and needs an “ether” for transmission.
Light is also a wave but doesn’t need an ether. Is light in space “wave” or “particle”? because when light bends do particles bend or this this one of the examples of light as wave?

Answer:

Sound is a pressure wave. It is the compressing of molecules. It is the compression that is the sound.

Light is a wave of electromagnetic fields, fields in this case being force representations. The light wave is a constant of nature in which a electric force changing will produce a magnetic force which will cause an electric force in a stable, repeating way traveling forward. There are other places where electricity creates magnetic forces and vice versa, but they are not stable and not self-contained.

When material is present, those electromagnetic forces exert themselves into a single point, which is what is classically defined as a particle. Light in space is then always a “wave”, since that is the way it propagates itself and travels.

When light is in a perfect vacuum, its speed is constant at c. When light is traveling through a material, the speed is slower than c, based on a calculation related to its “index of refraction”. This quantity is the change at which an electromagnetic wave will move forward while remaining self-contained and stable.

The ‘index of refraction’ is the change in velocities between material boundaries causing an apparent difference in the way that light is ‘seen’ (this is velocity that have direction now, not speed which is a directionless quantity). This is the reason you put a pencil in a glass of water, it seems to fracture or “refract” at the air-water boundary since the speed of light in each material is different.

So light never “bends”. What light bending is usually described that way because it is passing through things in which the velocity of the electromagnetic wave is different. This is the case of seeing mirages of water when the light from the sky bends up; the index of refraction of the air is temperature dependent.

how i saved my windows xp installation with knoppix

so its true. the difference between a ‘digital native‘ and a ‘digital immigrant’ is the amount of information that they contribute to the world. I absolutely hate having to wade through google searches to find an answer to my problems, so I try to post here about some of my tech adventures (even though it might be nerdy) hoping that one day a spider would come and crawl/index it and provide answers to people for the same questions I couldn’t find answers to!

So I tried to upgrade the ATI Radeon drivers on my Windows machine for my old video card (thinking it would improve performance! hah! no.) but I could not get Direct3d to work for my ol’ Radeon 9500. After the 26 or so re-boots fiddling with the hardware, the hardware connections and the hardware drivers, something finally happened to my Windows XP OS partition. It got fried.

I tried to boot up, but after the BIOS loaded I would get the error:

Windows XP could not start because the following file is missing or corrupt: \WINDOWS\SYSTEM32\CONFIG\SYSTEM
You can try to repair this file by starting the Windows Setup program from original floppies or boot from CD-ROM.
Select 'r' at first screen to repair

I did searches all over the internets to fix my problem, that of a corrupt WindowsXP SYSTEM registry, which caused my XP installation to not boot. Microsoft has a whole knowledge base article about this exact issue! Well, thats fine… just load up your version of XP from a CD (or floppy) and hit ‘r’ to repair, just like it says. Wait, except I have a SATA drive… and there is no native SATA drivers for those on the Windows XP CD! So Windows XP install CD does not see a valid drive with Windows on it to repair!

Damn.

I love this install of Windows, this drive has been with me since pre-Service Pack 1 days. Lots of tweaks (which caused the problem?) and lots of customization has made it unlike any other windows install I’ve used over the years. I wasn’t about to just re-image the drive after backing up the data.

So I gave Knoppix a try. For those that don’t know what it is, it is a Linux Live CD (meaning you run Linux from RAM and swap space) and supposedly contains lots of windows repair utilities. Booting it up, it had no problem seeing the SATA drives, booted into the OS and everything was fine. I was using the newest version, version 5.1.

I found the backup system registry ‘hive’ files at /System Volume Information/_restore/[Text String] and copied them to the desktop. Renamed them to be the operational one, but the Knoppix disk would not let me write back to the hard drive!

There is a problem with NTFS. The NTFS file system is a Microsoft proprietary OS whose protocols have been reverse engineered by the open source community. There are a couple different ways to make the NTFS formatted hard disk available.


mount -t ntfs /dev/hda1 /mnt/c

Which is the approximate default command in the /etc/fstabBut this command only allows the NTFS disk to be read, since the mount command (even with the -rw option) doesn’t allow NTFS partitions to be written to. Everywhere on the internets talks about this, that there could be problems with corruption if users were allowed to write directly to the file system using the mount command. So I couldn’t fix the registry hive this way.

So I went into alternate mounting techniques to allow the partition to be written to. Supposedly there is a program called CaptiveNFS (captive-ntfs) that is available on older versions of Knoppix, but wasn’t availiable on 5.1. So I downloaded Knoppix 3.6, only to find out that CaptiveNFS wasn’t supported any longer and did not work with Windows XP SP2. Shucks, again.

The problem, of course, was information overload.

Of course, Knoppix 5.1 came with utilities to write to an NTFS partition, it is noted as being the distro that allows users to recover windows partitions! There are two utilities: ntfschdsk, ntfsmount that do what I need: check the disk for corruption (and auto-correct things if it can) and mount the NTFS partition to write.

Running ntfschdsk I received:


CHKDSK is verifying indexes (stage 2 of 3)
Deleting index entry .DEFAULT in index $I30 od file 30.
73 percent completed.

So at least one entry was corrupt on the disk. Not a problem though, remount the partition with ntfsmount command. Then went into the previous registry save state and copied the following files DEFAULT, SAM, SECURITY, SOFTWARE, SYSTEM from /System Volume Information/_restore/[Text String] (where Text String is some naming auto-archiving naming convention) to the /windows/system32/config directory, overwriting the corrupted hive files.

Reboot the machine, the machine reads in the non-corrupted backup Registry Hives. Instant (instant being over the course of 3 days) success!

Hopefully, someone finds this post on the 5th page of google searches and finds what they need.

Originally written 12/10/2007 – finally posted 03/10/2008!

The Internets Is Scary Sometimes

Sometimes, you think that you are covered on the internets (all of them!), but then I get an email like this to my primary account:

As a courtesy, we are notifying you that XXXXXX users have found the following accounts for you:

   Flickr tmarthal on Flickr
   Digg tmarthal on Digg
   MySpace birddog on MySpace
   Picasa marthaler on Picasa

If you would like to make these accounts private, please
change the privacy settings on the original network and
XXXXXX will update its search results to reflect your changes.

To find your friends on XXXXXX, signup now.

I removed the company that sent the email, not sure that I want to encourage this type of email and account harvesting.

The point is, that someone, somewhere has correlated my different accounts on my various networks to my single email signon. Someone, somewhere knows my that the articles dugg on digg are associated with the pictures that I post on flikr! So, when I don’t post anything, they can check my pictures to find out what I was doing!

They missed twitter, delicious, slashdot, facebook and this blog though! And all of my troll accounts! Thats tood to know that there is some anonymity!