What is Standard Procedure?

At the moment, I have seven freelance clients. I've tried to move them on, explaining that my time is now very limited^[1] and I will be very slow to respond to requests. But they all wanted to stay^[2].

So I spent very little time maintaining these apps, making quick and dirty changes when they had feature requests. And the systems are suffering - they look old and clunky and the code is messy with a growing untested surface area.

The order-processing system

Then, one of the clients, who run a small, specialist, online store with a complex order-processing workflow, asked me about rebuilding their system. The front-end was in Wordpress/Woocommerce - they had asked an agency to build it for them and it was a mess^[3]. The back-end system was a very old Rails app that I had inherited. There are large parts of the code that are "over-engineered"^[4] with other parts where you get the feeling the original developer got bored and just cut corners.

I was playing around with Trello and realised that their order process could be handled quite simply as cards moving across lists. We just needed some way of asking for the right data at the right time and preventing certain movements, based on the values in that data.

And then I had my lightbulb moment.

At least four, maybe five, of my clients' systems could all be reworked the same way.

The clients would get a brand new system which shares bug-fixes and features. And I would get one code-base to maintain, not four.

Workflows

The light-bulb moment was about Trello and "workflows".

Standard Procedure will have "cards" moving through a "workflow" (think Trello's boards). Each workflow is divided into "stages" (lists). But unlike Trello, you can't just move a card between stages whenever you want. Instead, each stage defines the "options" available to you; you select one (and it may ask you for more data) and the option then triggers some "automations" (one of which will move your card to the next appropriate stage).

I mentioned earlier that Collabor8Online have already benefitted from my freelance work - well, shortly after I came up with the idea, we had a feature request which ended being an implementation of this very idea. And it works really well.

Folders, documents and forms

Another of my clients is a health and safety company. Their system is about recording "risk assessments" and "method statements", as well as making various types of documentation available to clients and warning them when that documentation needs renewal.

The majority of those documents are either standard files which are available to be downloaded (think standard policy documents) or forms that need to be filled in and stored for reference (like risk assessments).

So Standard Procedure will have a "file-system" - a hierarchy of folders which can contain documents. Documents themselves are either files (uploaded by the end-user or by the administrators) or forms (defined by the administrator and filled out by the end-user). Documents can also be entered into workflows (for example, if the submitted form needs to be approved by someone else). Finally, I want to have "slots" - a placeholder within a folder that someone needs to upload a file into. Once uploaded, that triggers an automation or workflow that could trigger an expiry notification in a years' time. This could be for a sub-contractor to upload their certification and ensure that it is always in date.

Timecards and internal processes

I have a retailer client whose system is primarily for HR but is actually used to run big chunks of their business. Managers at their stores build a rota, assigning shifts to various staff members. Staff members check in and out of work, and if they are repeatedly late or don't turn up, the system fires off warnings and possibly triggers a disciplinary process.

Things like new starter contracts, rota management, disciplinaries and appraisals are all perfect fits for the workflow idea. However, shifts and meetings (appraisals, performance reviews and disciplinaries) aren't simple documents - instead they represent spans of time.

Jobs

One of my clients is a staffing agency. Their customers publish jobs ("we need a qualified X to work with us for five days from the 8th to the 13th"), the staff who qualify receive a notification and can choose to take the job. Once the job is assigned, it follows a similar process to the retailer; the staff member records a timesheet, which gets approved by the customer and then marked for invoicing and payment.

Again, this is a very similar concept to shifts and timesheets managed by workflows, with just a few different rules.

Core concepts

So Standard Procedure will model the following things:

• People (I don't like calling them "users")
• Organisations (each of my clients have numerous clients or departments to manage)
• Permissions (probably organised via "user-groups" so we can control who can do what)
• Automations (with a variety of triggers and sequences of actions that occur when triggered)
• Workflows, workflow-stages and workflow-options (with optional questions that need to answered before an option can be triggered)
• Folders, documents, slots, forms and form-templates (where the templates define the structure of the form)
• Meetings, shifts and time-cards
• Activities and the audit trail - as soon as you have stuff happening automatically, we need to ensure that the client can trace through the changes and understand why things have happened the way they have.

The process

To build this, I'm going to follow an "inside-out" style. I'm going to pick a "story" and implement it from the point of view of an end-user - starting with the user's actions and working inwards to the database.

However, full-stack user-interface testing is a pain which often leads to failing tests because of timing and browser issues. Plus I suspect each client may need a customised user-interface to fully reflect the way they expect to see things.

So instead, I'm going to write specifications for the API - as if the system was being driven by an external system. Our spec will:

• set up a particular configuration
• simulate a person logging in (using OAuth)
• that person will manipulate the system using various API calls
• the person will read data back via the API
• the test will verify the data returned looks as it should

In other words, these will not be "unit tests"; we're not testing bits of functionality in isolation. These are "functional specifications"^[5] describing what someone using the system will see and do. Of course, along the way, we'll end up writing unit tests, but we need to be sure we're meeting people's needs, not just be sure that this component works in isolation.

Features and stories

A few of the stories and features I will need to implement:

Health & Safety: adding a new client

• given I am manager at a health and safety company
• and a set of folders containing standard policy documents is located within my account
• when I add a new client organisation
• then the policy document folders should automatically be made available, but read-only, to members of the new client organisation
• and I should be able to see that the folders were made available, read-only, in the system's audit trail

Retailer: recording a timesheet

• given I am a sales assistant working at a retail store
• and I have been assigned a shift on this week's rota from 10am till 4pm today
• when I check in at 10am
• then my timecard should be recorded as having started on time
• when I check out at 4pm
• then my timecard should be recorded as having finished at 4pm
• and my timecard should be marked as completed
• and my manager should be notified that my shift has finished
• and my manager should be able to see my shift within the system's audit trail

Online store: receiving a priority order

• given I am a manager at an online store
• when a priority order of type X is received in the "Orders" folder
• then my colleague should be notified that an order has been received
• when 4 hours have passed since the order was first received
• then I should should be notified that the order requires processing urgently

Staffing agency: client publishes a job

• given I am a manager at a staffing agency
• and we have a client organisation based in "Glasgow"
• and we have a number of staff members based at various locations around Scotland
• when the client user logs in
• and they create a new job document in their "Jobs" folder, listing the qualifications required for their job and the location where the job will take place
• then all staff members within 25 miles of the job's location, who have the required qualifications, should be notified about the job posting

Note that none of these make any reference to code. I could show them to my clients and they would understand what they are meant to achieve (even though, in reality, the clients aren't really interested, beyond the original discussion).

You might recognise these style of documents. The Agile Manifesto people call them "Acceptance Tests". Others call this "Behaviour Driven Development". I think of these documents as specifications - and we're going to use them to drive our development process.

By starting from the "outside", in terms that make sense to the people using the system, and working "inwards" into things like services, models and tables, we can ensure that what we actually build meets the customer's needs in a way that they can understand. And, I've found that when you work this way, you actually end up writing less code. Each specification is focussed on one task alone - we do the bare minimum to make it work, tidy up, then stop. And the best way to avoid bugs and complexity is to write less.

So that's what I'm building. Let's get started!