Agile Software Development

On one of the scripting groups yesterday, there were a few comments about Agile. One was to the effect that someone looking for a scripter wanted one who could take an Agile approach to trying things and improving the product dynamically. The other comment was that Linden Lab is using Agile, which explains the quality of their recent releases. I’m not sure whether that person was referring to how good releases have been, or how bad. Nor am I sure how much Agile LL is actually using.

I do know a bit about the subject and today I’ll talk a little bit about what I’ve learned and how it applies to my work here. Maybe what I say will give you some useful ideas. I hope so, certainly.

I’ve taken some courses from early members of the Agile software development movement, read their books, worked on some very good Agile projects, and even given a few talks at conferences. And I’ve helped some people improve their software development a bit … or at least they said I did. Not to toot my own horn, I’m certainly not Queen of Agile, but I do know a little bit.

Now there’s a lot of bad Agile out there. Big companies imposing some heavy process on everyone, companies of all sizes that don’t really follow the advice of the experts, and instead use “Agile” as a way to pressure developers for more work. I don’t support that kind of thing and I wouldn’t work for a company that did it. I’m here to talk about “Good Agile”, and really only the “how I develop software” part of it.

A basic idea of the original so-called Agile processes¹ is that you manage your work around a real piece of software that, while maybe it doesn’t have all the features you want yet, has all the features that it does have finished, well built, well tested, and ready to go. In a real business project, that keeps the discussion grounded in reality, with an understanding of what is done, what is not, and how long things are taking so far.

Not all of that applies to Second Life, at least not always. But one part really helps me and if you think it would help you, give it a try. And feel free to get in touch with me to ask questions if you want to. I love to try to help.

Anyway, sorry for all the introductory lead in, here’s what I do.

Have something working ASAP and always

I try always to have something that works. For example, when I started doing my Key-Value thing, I went as quickly as I could to get an AWS API talking to a DynamoDB database, into which I had put one record from the Amazon console. Then I wrote a tiny LSL script in Second Life to read that record. I suppose it took me two days to get the record in there, as I stumbled through the AWS jungle, being bitten by snakes and stepped on by elephants², but as soon as I had it, I made my record-reading script work in SL. It looked like this:

// http access to AWS API Gateway
// JR 2018-08-01
// /read?key=bill&aux=001

string Short_API = "https://REDACTED.execute-api.us-east-1.amazonaws.com/vt-kv-1/";
key Query;

default {
  touch_start(integer num_detected) {
    string q = Short_API + "read" + "?table=vt_pk_sk_table&pk=bill&sk=001";
    Query = llHTTPRequest(q, [], "");
  }

  http_response(key request_id, integer status, list metadata, string body) {
    if ( request_id == Query ) {
      string r = llDumpList2String([status, body], "\n");
      llOwnerSay(r);
      string v = llJsonGetValue(body, ["Item", "value", "S"]);
      llOwnerSay("value is \"" + v + "\"");
    }
  }
}

The output looked like this:

200
{"Item":{"value":{"S":"this is bill's first record"},"vt_sk":{"S":"001"},"vt_pk":{"S":"bill"}}}
value is "this is bill's first record"

Always tiny steps

I work always with tiny steps like that, each one tested.³

Why do I do that, when it’s obvious that my little first reading program isn’t anything much at all like the real system I’ll need? There are a few reasons:

• It gives me a jolt of satisfaction. When I get something working, I always feel good. Sure, it’s just a tiny bit, but I’m happy to get a tiny bit of satisfaction quite often, because it doesn’t take away from the big jolts when something big comes together after a while.
• I find bugs faster. Some bugs show up early on, but a lot of them don’t show up until we begin to integrate our code at a higher level. So the sooner I get started with that, the sooner I learn. That first sample didn’t work the first time. What I learned was pretty dumb: you have to deploy your API before its accessible. It’s best to learn that early and add it to my process.
• I find out what I hate faster. As soon as I had to use my API from a Second Life script, I started to learn what a pain it was to create the query strings needed. That got me started on a few little experiments to make it easier. Making it easier to write code helps me write code that’s more likely to work.
• I think about different issues. Writing my first little experiment and then my second one, got me thinking about security, and how I could best protect my data tables from malicious or accidental damage. I now have a decent idea or two on how I’ll do that, and a prototype version in hand.

So even though I don’t have managers and stakeholders who need to look at ongoing progress, I benefit from building in small steps myself. And when I don’t work in tiny steps? Often things go just fine. Sometimes, though, I find myself debugging for hours. When I go in small steps, that happens less often. I have no doubt that, for me, small steps are best.

But wait, there’s more.

Continuous testing

Agile experts suggest testing, and the grown-up languages have testing frameworks you can use, so that you can make a statement about what the program is supposed to do, and then check that statement to see if the program does it, quite automatically. Then, as you add features and improve the way you do things, you can run the tests again and be pretty certain that you haven’t broken anything. Or, if you’re like me, find out what you did break, and fix it.

For the K-V, as you’ll see in a later article, I have a bunch of tests written, that read records by key, write records, delete records, and do queries, checking each time to be sure I get back what I expect. All I need to do when I make an enhancement (I should say change since not all my changes turn out to be better), I run the tests and get immediate checks on how it’s going.

Now LSL is terrible for doing automated testing. Everything takes place in separate events, so you need some kind of tracking mechanism. And, unlike every language since before I was born, LSL has no ability for a program to look at the program. In other languages, you can load a testing library and from then on, any function you name test_XYZ or test_fred will automatically be called when you run the tests. So you don’t have to do a bunch of tedious wiring to write a new test.

That hassle means that even though I’m pretty good about having automated tests in Real Life™ languages, I’m not so reliable in Second Life. But I do have another habit that helps me a lot: I write tiny little test scripts for anything I’m not dead certain about. If I’m building a big script and need some new capability, and I’m sure how to do it, I build it in. But if I’m not sure (like maybe if it uses rotations) I’ll build a little test script, sort out the details there, and then import what I’ve learned into the big one.

Doing that lets me focus on just how to do the new thing, without having to juggle around all the other things the big script is doing. And, usually it gives me a better design for the new thing, since it’s all isolated and clean and I can see it.

Many small functions

What else? Well, I use a lot of functions. I was taught that if a chunk of code does some identifiable thing, don’t just comment it saying // does that thing. Instead, make a function called do_that_thing, and then call it. Two good things happen when I do that. First, my code is easier to understand. Reading a function that says do this thing, then do that thing, is easier than reading all the inline code to do those things. If I want the details, I can look. More often, it’s enough to know what’s being done and I needn’t worry about how. Second, it makes my code more modular, and modular code, for me, is easier to change and easier to use bits again.

Refactoring to improve the design

Finally, refactoring. Refactoring is the process of improving the design of the code without changing (that is, breaking) what it does. An example of a simple refactoring would be renaming a variable to make more sense. Maybe change it from J to Number_of_cats, I don’t know. With a decent text editor (I use Sublime), it’s easy to make that kind of change without breaking things. It’s especially easy if things are modular, as it turns out. Easier if I use lots of little functions.

What about efficiency?

One side note, about “efficiency”. Some folks ask me, but isn’t it more efficient to put the code in line instead of calling a function? Well, yes, it is more efficient. And if you can measure the difference between doing it with a function and doing it in line, you are a very good scripter. The difference is very small, and it’s swamped, for example, by the time it takes to run the loop you use to measure it.

What makes for efficiency is to use better algorithms, not fancy tricky code. And good algorithms are easier to put in when they are packaged up as functions. So it all hangs together. And I care about my efficiency more than the computers, and clear modular code makes me more efficient, and happier.

Summing up

So, summing up, I work in very small chunks, that work end to end. I test frequently, automated or manually. I write code that’s as clear and modular as I can, and I improve it often.

It works for me and maybe this will give you some ideas that work for you. What works for you will be different, but it’s worth trying ideas to see if they fit for you. And, again, feel free to catch me in world and ask me questions!

Bye for now!