Tuesday, 3 May 2011

MongoDB ASP.NET Session State Store Provider

Please note: v1.1.0 of this Sesssion State Provider is now available. See blog post: http://www.adathedev.co.uk/2013/03/mongodb-aspnet-session-store-provider.html
I've pushed up an initial drop of a custom ASP.NET session state store, backed by MongoDB, to my GitHub repository. After a quick test it seems OK, though could do with a real hammering before being deemed production worthy! So if you want to use it, just make sure you give it a good test through and most importantly, let me know how it goes! It's based on the sample custom session state provider in this MSDN article, basically just with the MongoDB specific bits swapped in.

Session state is stored in a "Sessions" collection within a "SessionState" database. Example session document:
{
    "_id" : "bh54lskss4ycwpreet21dr1h",
    "ApplicationName" : "/",
    "Created" : ISODate("2011-04-29T21:41:41.953Z"),
    "Expires" : ISODate("2011-04-29T22:01:41.953Z"),
    "LockDate" : ISODate("2011-04-29T21:42:02.016Z"),
    "LockId" : 1,
    "Timeout" : 20,
    "Locked" : true,
    "SessionItems" : "AQAAAP////8EVGVzdAgAAAABBkFkcmlhbg==",
    "Flags" : 0
}

Inline with the MSDN reference ("ODBCSessionStateStore" changed appropriately to "MongoSessionStateStore"):
If the provider encounters an exception when working with the data source, it writes the details of the exception to the Application Event Log instead of returning the exception to the ASP.NET application. This is done as a security measure to avoid private information about the data source from being exposed in the ASP.NET application.

The sample provider specifies an event Source property value of "MongoSessionStateStore". Before your ASP.NET application will be able to write to the Application Event Log successfully, you will need to create the following registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Eventlog\Application\MongoSessionStateStore

If you do not want the sample provider to write exceptions to the event log, then you can set the custom writeExceptionsToEventLog attribute to false in the Web.config file.

The session-state store provider does not provide support for the Session_OnEnd event, it does not automatically clean up expired session-item data. You should have a job to periodically delete expired session information from the data store where Expires date is in the past, i.e.:
db.Sessions.remove({"Expires" : {$lt : new Date() }})

Example web.config settings
<configuration>
  <connectionStrings>
    <add name="MongoSessionServices" connectionString="mongodb://localhost" />
  </connectionStrings>
  <system.web>
    <sessionState
        mode="Custom"
        customProvider="MongoSessionStateProvider">
      <providers>
        <add name="MongoSessionStateProvider"
             type="MongoSessionStateStore.MongoSessionStateStore"
             connectionStringName="MongoSessionServices"
             writeExceptionsToEventLog="false"
             fsync="false"
             replicasToWrite="0" />
      </providers>
    </sessionState>
  </system.web>
</configuration>

A few points to note, to give some control over fault tolerance and consistency:
  • if you want updates to the MongoDB store to be fsync'd before returning, set fsync="true"
  • if you have a replica set and want updates to be persisted to multiple nodes before returning, set the replicatesToWrite setting to the appropriate number.
SafeMode is currently always enabled.

If you give a whirl, please let me know how you get on.

29 comments:

  1. Awesome - wonder where you would get a crazy idea like that? :)

    Sorry I didn't have the chance to help more with this...

    ReplyDelete
  2. @Matt - :) No need, to get it to this point was really not much to it. Turned out there was some more interest in this too.

    ReplyDelete
  3. Up and running in like 5 minutes. Thanks for putting this out there. I'll keep you posted on how it's working for me.

    ReplyDelete
  4. @Aaron - Thanks for taking the time to let me know - please do, I'd be interested to hear how it goes!

    ReplyDelete
  5. Nice post !

    It's up and running for me too.

    For you, what is the main pros to use MongoDB instead of, for example, Sql Server mode as session provider?

    We are using SqlServer for our app (Amilia.com) and we follow with interest all .NET codes using MongoDB, it should be nice to have your opinion on that topic...

    ReplyDelete
  6. @dervalp - thanks! I guess the main difference is cost - obviously you need a license for SQL Server whereas you don't for MongoDB. If you have a very high traffic site, with a high level of session activity, then MongoDB could be an option to keep costs down. It's all about using the right technology for the right purpose in the right scenario.

    ReplyDelete
  7. Congrates to you for winning the Microsoft Community Contributor Award (2011).


    I had also recevied MCC recently :)
    http://tnvbalaji.wordpress.com/2011/04/28/microsoft-community-contributor-award-winner/

    ReplyDelete
  8. @tnvbalaji - Thankyou, and to you too!

    ReplyDelete
  9. Thanks very much for this provider. It worked immediately for me. I'll let you know if I do run into any issues. Are you aware of any areas of weakness, or is the health warning just precautionary?

    ReplyDelete
  10. @pete - thanks for the comment, glad you got it working without any hassle! It was precautionary as I haven't stressed it to any real degree. The only point, which I raised above, is about clearing out old sessions. Let me know how you get on

    ReplyDelete
  11. I am giving your implementation a whirl. I added a an encryption layer because of a requirement. So far so good.

    ReplyDelete
  12. Did any one test it into cluster configuration? How does it handle concurrency when requests for a user using Session object are fullfiled by different nodes in a IIS webfarm?

    ReplyDelete
  13. One failing to point out with this code is items added to the TempData collection aren't being removed at the end request.

    ReplyDelete
  14. @Jake - Yes, I did mention that in my post - you'll need a job to run periodically to remove old sessions

    ReplyDelete
  15. @Adrian... I am coming back on handling concurrent requests for the same session id. As you know, Mongodb does not have transactions support. In your code (SetAndReleaseItemExclusive, GetSessionStoreItem) I find a lot of operations like Update, Remove and Inserts which assume that the document will not be modified between two consecutive operations with Mongodb. Did you take into consideration in which documents might be changed between two mongo operations? I am trying to figure out if your Mongodb session state store is a good choice for having a fast and replicated solution for a farm of IIS nodes. InProc ans Session state service are not a good fit for a webfarm and SQL Server session store is much too heavy and expensive in HA setup.

    ReplyDelete
  16. @Robert - Note, in the case of SetAndReleaseItemExclusive, in the update code-path, there's only a single (atomic) update happening. In the insert code-path, yes there are 2 operations (delete, which is a "just in case the session does exist" followed by an insert). It's the same as the Sample Session-State Provider given on MSDN (ODBC, Access-backed), which doesn't use transactions.

    But TBH, for use in such an environment, my answer has to be that I would recommend trialling it/run a full stress/functional test. As I haven't used it in such an environment and can't claim to have completed a thorough stress test.

    ReplyDelete
  17. Not to nit-pick, but I notice you use DateTime.Now.ToUniversalTime() often. Why not use DateTime.UtcNow instead? As I understand it DateTimes are UTC under the hood and performs a translation based on the system settings for the local time zone every time "Now" is called, which is then turned back into UTC. Also, not that it really matters in this case, but if the system is under stress, a few milliseconds may pass between the calculations of each step and you could be off by 1 or two ms.

    I only bring this up as I'm looking for a high-performance and stable session state solution for a web farm

    ReplyDelete
  18. @Mizchief - No reason behind the use; don't even remember it being a conscious decision :) I would go with DateTime.UtcNow now, but doubt it would make any performance difference.

    ReplyDelete
    Replies
    1. No problem man, the code has been working great so far, just adding a few tweaks here and there for re sharper suggestions. Right now our production site has about 10,000 active sessions so every little bit helps :) I read somewhere that DateTime.UtcNow is about 30 times faster than DateTime.Now. Also, when using mulipule date settings in the same function it's good to call UtcNow once, and stash in a local variable to use during the rest of the function. Potentially the Created, Expires, and LockDate can be off by a few ms which could cause an issue down the road if someone is counting on them being the same.

      ex:

      public override void CreateUninitializedItem(HttpContext context, string id, int timeout)
      {
      MongoServer conn = GetConnection();
      MongoCollection sessionCollection = GetSessionCollection(conn);
      BsonDocument doc = new BsonDocument();
      doc.Add("_id", id);
      doc.Add("ApplicationName", ApplicationName);
      DateTime dateTime = DateTime.UtcNow;
      doc.Add("Created", dateTime);
      doc.Add("Expires", dateTime.AddMinutes(timeout)); // DateTime.Now.AddMinutes(timeout).ToUniversalTime());
      doc.Add("LockDate", dateTime);
      doc.Add("LockId", 0);
      doc.Add("Timeout", timeout);
      doc.Add("Locked", false);
      doc.Add("SessionItems", "");
      doc.Add("Flags", 1);

      try
      {
      var result = sessionCollection.Insert(doc, _safeMode);
      if (!result.Ok)
      {
      throw new Exception(result.ErrorMessage);
      }
      }
      catch (Exception e)
      {
      if (WriteExceptionsToEventLog)
      {
      WriteToEventLog(e, "CreateUninitializedItem");
      throw new ProviderException(_exceptionMessage);
      }
      else
      throw;
      }
      finally
      {
      conn.Disconnect();
      }
      }

      Delete
    2. Thanks, it's great to have the feedback! Re: resharper tweaks, yes it is a big frustration of mine, not having Resharper on my dev pc at home!

      Delete
    3. Glad to help!

      Anyone have a batch file handy that can be run on any server in the replica set and then find the primary and run the delete expiration?

      I'll probably just make a console app to do so since the driver makes this easy, but if anyone has this hand it could save me the trouble.

      Delete
  19. Thanks so much for sharing your code. I gave it a go with some stress testing, and the only problem I ran into was that if you Disconnect after each use of a connection, then you run out of sockets. The C# Mongo interface handles connection polling internally; manually disconnecting connections prevents this pooling from working. Simply removing the finally blocks with the disconnects solved that problem.

    ReplyDelete
  20. Hi,

    thank you for sharing this. It's been really helpful in our project.

    We've found through load testing this code has the following problem: it calls the "disconnect" method on the server object in several places. This may cause the Mongo driver to run out of connections and fail to connect to the server.

    It turns out that the disconnect should not usually be invoked; it's better to let the driver handle the connection pooling internally.

    See for example this blog post: http://craiggwilson.wordpress.com/2012/09/23/disconnecting-with-the-mongodb-driver/

    We removed the "disconnect" from the finally blocks and we got rid of Mongo drivers errors on connecting with high load scenarios.

    ReplyDelete
    Replies
    1. Thanks Stefano, good to hear how you've got on with it. If you could raise this as an issue on the github repository, that would be great (https://github.com/AdaTheDev/MongoDB-ASP.NET-Session-State-Store) - I'll have a look into it and push up a fix. Cheers!

      Delete
    2. This has now been addressed - https://github.com/AdaTheDev/MongoDB-ASP.NET-Session-State-Store/issues/2

      Delete
  21. Do you plan to update with new Mongo drivers? The current drivers (1.7.0.4714) have deprecated some of the functions that you are using, i.e. SafeMode => WriteConcern, Connection => MongoClient, etc.

    Thanks
    Nate

    ReplyDelete
    Replies
    1. Yes, it's been on my todo list for a bit. I'll bump it up the list. Watch this space :)

      Delete
    2. I've now pushed updates to GitHub to use v1.7.0.4714 of the drivers.
      http://www.adathedev.co.uk/2013/03/mongodb-aspnet-session-store-provider.html

      Delete