Key-Value? What and why?

Let’s talk a bit about key-value storage. What is it, why have I chosen it, what’s it good for, how might one use it?

A key-value store is like a hash table, hash map, or map, if you’re familiar with those terms. There are keys, and values. In our case, both the keys and the values are strings. Each can be any string you want to assign.

So we might have the key “janet”, with the value “friendly neighborhood scripter”, and the key “dizzi”, with the value “inquisitive tester”. If we tell our key-value store “remember ‘janet’ is ‘friendly neighborhood scripter'”, it will remember that until we tell it to remember something different for the key ‘janet’.

Now of course we might want to remember things that make more sense, perhaps the scores people have made on various games in the region. So we could store, perhaps:

• janet-bingo: 500
• janet-sudoku: 3150
• dizzi-bingo: 1103
• dizzi-bridge: 1531

We’ve chosen the convention that we’ll store the first name, a dash, and the game name as the key, and their total score as the value. We could choose any convention that makes sense to us.

In our quest in Lexicolo, we store a player’s information under a key consisting of the string “quest_” followed by their avatar key, so that my information looks like this:

quest_22ba38c9-06e9-4d42-8a6d-495e35bc33b7:{"br_chain":1,"inventory":{},"money":253,"quest_level":"freja"}

As you can see, I’m not very far along in the quest, because I’ve started over, so I just have one piece of freya’s chain, 253 money and I’m in the first level, namely ‘freja’. Point is, though, we can find anyone’s current quest status by looking up their quest key, “quest_” followed by their avatar key, and looking at the data. We can then update the data, like give me more money, and put the information back into the key-value store.

The data format there, by the way, is JSON, which I’ve mentioned before. Our quest game code reads the player’s value string, turns that into whatever it uses inside the quest to keep track of money and level and so on, and then as the player progresses, we update our memory tables, create new JSON, and write out the player’s updated information.

My sister Dizzi is doing much better. Her current record is

{"inventory":{"apple":4,"mushroom":6},"money":41,"quest_level":"loki"}

This isn’t Dizzi’s first time through either. We presently have over 1800 people who got a quest HUD, and about 110 who have at least started our quest. Only about 28 have completed all stages of the quest: it’s not easy. And all of those people are stored in the Experience’s key-value datastore.

In case you’re wondering, we have about 220,000 characters of information stored, and the database’s total capacity is about 134,000,000 characters. So feel free to come by and play the quest! We’re not likely to run out of space for your information.

So a key-value store is rather rudimentary, pretty much limited to read a record, write a record, with a few other commands to find out what keys are there and such. But because its keys and values can be any string, it is very flexible, limited only by your imagination about what you want to store and how you want to represent it.

One more thing. In the new K-V thing I’m building on Amazon Web Services, the keys have two parts, the primary key and the secondary key. In my case those are both strings. The same primary key can be used with as many different secondary keys as you want, so you could have janet/bingo with my bingo score, and janet/quest with my quest record, and janet/pizza with my favorite pizza order. It’s all up to you. The AWS system allows you to read and write using both keys, so you could update my janet/pizza record if i suddenly start liking anchovies.

But you could also ask it to give you all the janet records there are and it would give you back all three of those. I’m setting it up to give them to you one at a time, because SL doesn’t allow very much data to come back from a web access. AWS could use any limit at all, and it’ll stop when the limit of records is used up, or when your available memory buffer fills up. Naturally, it gives you some data you can use to tell it where to pick up for the next batch.

Now I wrote about my project goals elsewhere, and I picked key-value because it’s simple, but flexible. SL scripts aren’t very big, and while using key-value will make the scripter think a little harder, there’s much less work to do configuring the remote data. Pretty much no work at all, in fact, as I’ll have set everything up, ready to go.

Well, that’s about all I have to say about K-V storage for now. If I think of more, I’ll write more, and if you have questions, leave them in the comments and if I’m still alive, I’ll get to them.

Thanks!