Tag clouds (and a lot of cryptography), part 7

After smashing my head against a cryptographic wall and going through some code samples, blogs and articles, I think I have finally solved my symmetric cryptography problem. Those who don’t know what I am talking about should probably read my previous posts on the subject – Tag Clouds Series. The programming language used is C# 2.0.

Before putting forth the solution, let me briefly state what I wanted to do.

• The application allows the user to create multiple blog accounts and accepts his passwords for each of them. When he uses the app, he only has to enter one master password and the app will communicate with his accounts using the passwords it already has.
• Since the account details are stored in an xml file, I have two options.
  1. I can decrypt and encrypt the whole file every time it is loaded and saved. The problem with this method is that the contents of the whole file will be in memory in the clear, including the passwords!
  2. I can keep each password encrypted with a separate key. In this case too, I load the entire file into memory. But I only encrypt/ decrypt passwords when actually required.

  Naturally, I prefer the second method. So does Keith Brown (refer to Tag clouds, part 6 for links to his MSDN articles and accompanying code).

Now, my unoriginal solution.

Setting things up

1. User provides a master password.
2. A random salt, known as the master-salt, is generated using the RNGCryptoServiceProvider.
3. An iteration-count is determined. It can be any integer >= 1000. It can be hard coded into the application.
4. The Rfc2898DeriveBytes class is used to generate a master-key. The class constructor takes the master password, salt and iteration-count as parameters. At this point the master password is no longer required. But since .net strings are immutable, I don’t know how and when the GC will collect the memory. Anyway, since that is out of my control, I simply forget about that. If anybody knows why I should not do that, please tell me.
5. Another key, known as a verification-key is derived similar to steps 2-4. The only differences being – the master-key is used as the password and the salt, called the verification-key-salt, is a known string hard coded into the application.
6. I now store the master-salt and verification-key in the xml file in a separate section.
7. Every time the user starts the app, it prompts for the master password. Once it is acquired, the app loads the master-salt from the xml file and goes through steps 4 and 5. If the computed verification key matches the one stored in the file, the password is correct. Otherwise it is not.

Encrypting and decrypting records

1. As far as a new blog account goes, I am only interested in keeping the password safe. Every time a new account is created, I generate a salt using RNGCryptoServiceProvider. This is stored in the file along with each record since it is a per record salt.
2. When the account password is provided, I take that, the master-key, record-salt and the iteration-count and send it to an EncryptData function.
3. The EncryptData function passes the master-key, record-salt and the iteration-count through the Rfc2898DeriveBytes class to generate the record-key.
4. This record-key is used to derive a 256-bit key and a 128-bit initialization vector for the Rijndael algorithm. The record-key is discarded after this.
5. The password string is converted to a byte array and is encrypted. The resultant byte array is converted to its base64 equivalent. This encrypted string is stored in the xml file with the record.
6. When the clear password is required, the encrypted-password, master-key, record-salt and the iteration-count are sent to a DecryptData function.
7. The record key is regenerated by the Rfc2898DeriveBytes class and the key and initialization vector for Rijndael are acquired.
8. The encrypted-password is converted from base64 to a byte array and is decrypted. The resultant byte array is converted to its text equivalent. This clear password is returned.

What happens when the user changes the master password
Well, very simple really.

1. I generate a new master-salt and master-key (don’t discard the old ones just yet).
2. Then, for each record-password, a new salt is generated, keeping the old one safe. The encrypted password is first decrypted using the old master-key, old record-salt and the iteration-count. Then it is encrypted using the new master-key, new record-salt and iteration-count.
3. When all this is done, I generate the new verification-key and save that and the new master-salt in the file. That is all.

The actual code that does all these things is spread over three different modules. If you want all that, you will have to wait for me to release the app and the source code. But I am providing the entire code of the CryptoHelper class in a text file – CryptoHelper Code. A few clarifications. One, I use two different iteration counts – 4096 for generating a master-key and 1024 for regular ones. And two, I understand that the Rfc2898DeriveBytes class generates the key using a PBKDF2 (Password Based Key Derivation Function 2) function which is specified in the RSA PKCS # 5 guidelines. Never read them. But you might find the information useful.

This and that
Okay, now that the technical part is done, I provide a few links to various write-ups I found helpful.

• Keith Brown’s two MSDN articles from 2004 and the source code provided by him are the basis for my code (Security Briefs: Mind Those Passwords! and Security Briefs: Password Minder Internals). But I have to say that I could not understand what was going on till I stepped through the code in the debugger.
• Cryptography 101 for the .NET Framework
• .NET Security Blog : Generating a Key from a Password
• I discovered Symmetric Key Encryption using Rijndael and C# when I was searching for information on whether it is safe to pluck the Initialization Vector from the PBKDF2-generated key. It is written for .net 1.1 and is good enough except for one serious flaw – the password is used as salt, which to me is very unsafe. It is basically a concatenation and defeats the very purpose of using a salt. A salt is preferably generated randomly (Salt (cryptography)).

Readers should note that the application is still under development and I might change the code a bit (or a lot) depending on what else I discover. If someone thinks that there is something seriously wrong with the code (as far as cryptography goes – don’t talk of coding style), I would appreciate a comment regarding the same.

Tag clouds, part 6

Well, how do you encrypt and decrypt data in .net 2.0 using symmetric cryptography and a password? It should be simple and safe, right? Doesn’t seem that way to me. I have limited experience of using cryptography in applications (just hashing and digital signature verification). So this is the question that currently has me stumped.

I thought you just have to provide a password and the plain text to some CryptoServiceProvider and the data will be magically encrypted. But you have to do more than that. First of all, a password is not sufficient; it has to be of the correct key size as required by the encryption algorithm (Rijndael and TripleDES being my choices) that you are using. Then, there is the question of the Initialization Vector. Another problem I am facing is how to store the password in memory if the application needs it for the entire session? Is it safe to keep it lying around? Surely I can’t ask the user to type in his password every time I need to load and save the data – browsers don’t do that and Thunderbird does not do that.

Well, those are some of the problems that need to be solved in my tag cloud generator. In fact I think I am close to a solution. Just thought I should write about the problems before writing about how I solved them.

These are a few articles available on MSDN on this very topic.

• Security Briefs: Mind Those Passwords! – Keith Brown
• Security Briefs: Password Minder Internals – Keith Brown

Tag clouds, part 5

Made further progress on the application. Have added ability to have multiple blog accounts. And it now does its work on a background thread. So the UI is more responsive. BTW, Jim is working on a Tag Cloud Generator of his own. It has not been released yet.

Tag clouds, part 4

I have been able to write a small software based on what I have written in Tag clouds, part 3. And it works decently. The software generates html code for tag clouds. But it is far from complete, yet.

I made another discovery yesterday. WordPress XML RPC Support provides links to various xmlrpc based apis. But the information is confusing. And links to the movable type apis are broken. Lorelle provides the correct link – Six Apart Developer Documentation – XML-RPC. There is a very interesting api – wp.editPage, which can be used to edit a page remotely. So, if I have a separate Tags page, I can make the software automatically update the page on every run. No manual copy-paste stuff needs to be done. There is a minor setup procedure which is required. But I find that acceptable. Checkout my tag cloud page which is edited remotely though the software. [Edit: Nov. '09; I don't find the separate page to be of much use. The wordpress tag cloud widget is sufficient for most needs.]

I have to sort out a few issues -

• The code is very brittle and needs cleanup. Even after that is done, I can’t say that it will work in all cases coz, being new to wordpress, I am not aware of how things work in different installations/ scenarios. All I can say it is that it satisfies my present requirements.
• Slug renaming needs to be implemented. Presently, it only replaces spaces with dashes/ hyphens while creating tag urls. I googled for any other requirements and have come up with the following information -
  • How to Write Descriptive Titles without Making URLs Too Long in WordPress
  • WordPress Post Writing Advanced Features
  • Understanding WordPress Permalinks

  As of now I have decided to allow only A-Z, a-z, 0-9, – and _. Spaces will be replaced by hyphens and multiple back-to-back hyphens will be consolidated into a single one. That is what wordpress seems to do. One more thing I found out was that slugs are limited to 55 characters.

• Another problem is that the apis do not expose the post status (draft, private, pending review, public). So there is no way to know if the tag cloud is pointing to posts that cannot be accessed publicly. This bug has been noted on WordPress trac and won’t be taken up till WordPress version 2.4. So, using the software with private or unpublished draft posts will give incorrect results.

That, I think covers most points. Let me see how things proceed from here and when I can provide the source code for download.

Tag clouds, part 3

I have been looking around for a tag cloud creator coz the existing widget does not update itself properly (the timing of the update seems to be random). There are a few issues in this, the first one being the confusion between tags and categories. Some off line tag cloud creators are available (such as Tag Cloud Generator). But for them, tag is another name for ‘category’, and rightly so coz tags are a very recent feature and the word was used interchangeably with ‘category’ till very recently as far as wordpress is concerned.

I did a little more research on xmlrpc and how it can be used with wordpress.com. 5 Advanced Techniques for Creating Plugins and Widgets for WordPress.com provides some interesting information. The primary source, however, is XmlRpc and XmlRpc Implementations. A Microsoft.Net XmlRpc class library is available with documentation courtesy Cook Computing.

I previously suggested that tags are not accessible via xmlrpc (I was using the metaweblog api from the above library). I was wrong. Tags are indeed supported (wordpress calls it ‘keywords’ though). Windows Live Writer as well as quite a few desktop blogging softwares support this.

So, finally, it seems that you have to write your own tag cloud generator. Since tags are supported, you can use the xmlrpc libraries to write a simple application that pulls all the posts, parses their tags property while maintaining a count, and then generates code in –
<a href=”tag-url”><font size=”tag-font-size”>tag-name</font></a> – form for each tag. But this will have to be done after every post or batch of posts or whenever you want to update the cloud. You will have to run the software, get the tag cloud code, and paste it in a text widget or on a dedicated page on your blog. Another disadvantage is that the time and bandwidth required to accomplish this will increase in direct proportion to the number of posts.

I will provide one piece of information – to access tags from the XmlRpc.Net library, copy the MetaWeblogAPI.cs file to your project and modify the Post structure by inserting public string mt_keywords; somewhere in the structure.

It works for me, but the method is nothing more than a kludge.