SimpleDBM - a NoSQL Transactional DB in Java

The goal of the SimpleDBM project was to primarily teach myself how DBMSes work. In that goal it succeeded I think, and it also was great fun researching all the computer science literature on database technology and applying the techniques invented by great pioneers in this area.

I highlighted some of the sources I used in the implementation of SimpleDBM in this blog post.

Unfortunately due to lack of time I have not been able to devote much time to SimpleDBM in the past few years. So new features are not being implemented, the project is in maintenance mode; that is, I will fix bugs reported.

It is not possible to know if anyone is using SimpleDBM or not. I have not used it in anger in a Production environment so in that sense it is not Production software. However, I think its main value today is pedagogical in that it can be used to understand and learn the traditional techniques used to implement database engines. The implementation is much better documented than any other opensource DBMS I have come across. This is partly because I once thought of writing a book on how to implement a DBMS.

The implementation handles some of the hard problems, such as transactions, write ahead logging, concurrent and recoverable BTREE operations, deadlock detection, etc.

The project is now hosted on GitHub.  For anyone just wanting to use it Maven packages are available.

What's great about Lua?

Lua is an amazing programming language implementation. I say implementation because it is not just the language itself but how it is implemented that is particularly impressive.

As a programming language, Lua can be characterised as a small but powerful language. The power comes from clever use of a few core meta-mechanisms as Lua authors like to put it. A nice introduction to some of these are in the recent talk by Roberto Lerusalimschy at the Lua Workshop 2016.

I used to think that Lua is a simple language; but appearances are deceptive. I now think of Lua as 'small' and 'powerful' language rather than a 'simple' language.

The language design is clever, but the implementation is what makes it great.

Firstly it is a very compact implementation, just a few C source files, and that's it. No dependencies other than an ANSI C compiler.

Secondly, despite the compact implementation, it features:

• A byte-code compiler and Virtual Machine.
• An incremental garbage collector.
• Extremely fast parser and code generator.
• And the language is delivered as a library with an extremely well designed C API, that makes it easy to embed Lua as well as extend it.

It is this combination of economical design and beautiful implementation that makes Lua great.

Lua 5.3 Bytecode Reference

Lua bytecodes are not officially documented as they are considered to be an implementation detail. The best attempt to document Lua bytecodes is the A No-Frills Introduction to Lua 5.1 VM Instructions by Kein-Hong Man. However this document is quite old now and does not reflect the changes made since Lua 5.2.

Some time ago I started an attempt to bring this document up-to-date for Lua 5.3. I recently managed to spend a few hours updating the new Lua 5.3 bytecode reference. This is still not complete but the most important bytecodes are covered.

I want to eventually produce a version that is like a specification, i.e., one that allows independent implementations to replicate the bytecode generation. My interest in this is due to my desire to create a new parser and code generator for Lua. 

Unique problems of a start-up

Well recently I took the plunge and started my own tech start-up after years of thinking about it. So I have been battling the usual things that many start-ups face I guess.

Of great help are inspirational talks such as these.

First is the talk by creator of Ruby on Rails - David Heinemeier Hansson at Startup School 08 where he talks about how to create a successful start-up.

Next is this talk by Walter Bright where he talks about how he started the programming language D, and he describes the way he works.

My journey has only just started. I would love to discuss some of the issues that start-ups face and how I am trying to solve the ones I face.

I love the new Microsoft!

I have always been a Java person ... until now. With the new cross platform open source CLR (.Net) platform from Microsoft, I am doing exciting new work in C#!

I also love the Linux subsystem on Windows 10 - which I installed today. I was able to build my Lua derived scripting language Ravi using Linux tools right within Windows. I did not enable LLVM but that is next on my list to try. If this really works, then I can decommission my Linux Virtual Machine.

I also love that Microsoft's made the Visual Studio Community edition free - and this is the full version and not a cut-down version. And the new Visual Studio Code editor is great - checkout my Lua/Ravi 5.3 debugger extension for it!

All in all - great stuff, Microsoft!  

Never been as good as now for creating software

When I started building software 25 years ago, one had to pay for everything, even a basic C compiler was not free. There was no internet, no Linux, no OpenSource.

The computing world has really moved on.

Now when one starts a project there is a whole range of OpenSource building blocks one can choose from. And really cool stuff too. In no particular order here are some of the projects that have excited me in recent times:

• CoreCLR
• grpc
• Protocol Buffers
• Apache Zookeeper
• RocksDB
• Lua
• LuaJIT
• Visual Studio Code
• libuv
• node.js
• LLVM

One could go on. The wealth of knowledge that is now accessible to all is tremendous, and it is all there for anyone who is interested. 

I think this is our time - we who create software. The world is being changed forever as software will drive everything everywhere, and we, the software creators, are at the core of this revolution.

From Java to C#

I am using C# for the first time in a major new project. This has been made possible thanks to Microsoft's open sourcing C# and making it cross platform. I am using C# both on the client desktop as well as the server backend.

My initial impression is that as a language C# is much more productive than Java!

If only Microsoft had done this earlier - the programming landscape might have been quite different today. Java still has an advantage of a larger eco-system in the opensource world, especially when it comes to writing server applications.