Categories:
Ludum
Ludum was a mobile-first price comparison app targeted specifically at video games and video game peripherals (such as controller), which I started in 2015 and later I added a full blown web application into the mix. I started this side-project in the end of 2015 and eventually killed it of in April of 2020. One of the features I was most proud of was integrating OCR support for reading barcodes and using that to display the data that the backend had on that game.
The software architecture and backstory
I started this as a project to learn more about native Android Development, Python (Flask) and MongoDB. The aim was to use a Flask application as “backend” and have it transform the data and expose this data to the mobile app through a JSON API.
But during the early prototyping phase, I realized pretty quickly that if I wanted to get something shipped (soon ™). I needed to use a different tech stack. I was having the hardest time just getting the basic things working, things like looking up records (or ‘documents’ as they are called in MongoDB) and exposing them in a decent fashion towards the mobile app through the JSON API. Or even working out the basic kinks in the mobile app.
I had some very minor experience in both native Android and Python. So after about 6 months, I decided I was gonna reboot the project with a different (and somewhat more familiar) tech stack. I’ve already been working as a PHP developer for a few years, so I decided to grab Lumen (and later migrated to full blown Laravel), MySQL and Ionic. Redoing the (back then state) project in Lumen and MySQL felt like a breeze. For the backend in 2 days I already had the same functionalities which in Python took me multiple weeks. So I initially continued working on the backend and get that into a state where building an app for, would make sense.
Once the backend was in a somewhat decent state I stumbled upon Ionic. For the people that don’t know, Ionic is a mobile application framework that is commonly referred to as a “hybrid app”.
That means that the framework builds a native binary with just a “WebView” component in it and that WebView you runs your HTML, CSS and JavaScript code. It also provides you with the capabilities to speak to certain pieces of hardware, like the camera, NFC and Bluetooth all through JavaScript. That previously mentioned binary can then be distributed to users through Google’s Play Store and Apple’s App store.
Ionic back then (Ionic 3 to be exact) only had support for Angular and PhoneGap / Cordova plugins to speak to the hardware natively. Which back then could definitely cause some weird build errors, since some Cordova plugins could cause some funky conflicts during the build process. But in 2 days, I had a somewhat decent working prototype and 3 days later my app was getting into state where I would feel comfortable using it myself.
So with that all out of the way, I launched my first public version of the app in 2017 and the rest is history :)
What did I learn from it, from a technical viewpoint
From a technical and software architecture point of view, I’ve learned allot from this project. I learned that in projects like this the following things;
- Version your API’s
- It’s absolutely necessary to have automated testing
- Don’t assume your API is always available
Version your API’s
Always version your API. If you choose to use /api/v1/some_endpoint, /api/v2/some_endpoint, /api/v1.1/some_endpoint or some other variation. But once you deployed that API to production and users are using that API, it is basically (within reason) set in stone. You can add new fields (which older client’s don’t depend on) but removing used fields is most likely out the question.
Since it’s possible that you run into the situation where you just released version 1.3.0 of your app, but the majority of your users haven’t updated yet from the current 1.2.4 version. That might be due to the fact that the new version of your app has just been pushed to Google’s Play Store and Apple’s App Store, because (some) users still hasn’t updated or simply because some users aren’t capable of updating to the latest version of your app because requires a new(er) version of Android or iOS.
So for example; say you are building an weather app with the following JSON API. It has a basic endpoint which the client can talk to based on the following format /api/v1/location/<city_name>,<country_code>. An example API request could be /api/v1/location/amsterdam,nl and the API would return the following:
{
"conditions": "sunny",
"temprature": "18.0",
"temprature_measurement": "celsius",
"humidity": "40",
"humidity_measurement": "%"
}
But say you deployed a brand new version of your backend which output the following JSON on the same API request:
{
"conditions": "sunny",
"temprature": "18.0",
"humidity": "40",
"measurements": {
"temperature": "celsius",
"humidity": "%"
}
}
What will happen, if an older version of the app hit that API and tries to access properties that doesn’t exist anymore or if a newer client accidentally hit’s a backend server where the ‘old’ backend code is still running. The client / app will most likely chrash.
These are all situations which you as a (backend) developer need to take into consideration when you want to make changes to an existing API and where you as a developer don’t have full control over the consumer of the API (or force users to a newer version).
In this example, if you really wanted to keep the same endpoint but also support the ‘old’ and ‘new’ apps, I would go with an approach like this. It definitely looks a bit bloated and contains duplicate information (temperature and humidity measurements) but it’s better then apps that are crashing 😉.
{
"conditions": "sunny",
"temprature": "18.0",
"temprature_measurement": "celsius",
"humidity": "40",
"humidity_measurement": "%",
"measurements": {
"temperature": "celsius",
"humidity": "%"
}
}
However, ideal case scenario I would implement a new endpoint called /api/v2/<city_name>,<country_code> where I would return the JSON the newer clients depends on.
That means you also need think about the length of support of your API’s. In a corporate environment, management will most likely tell you to “support it until the end of time” (paraphrasing of course). but that’s definitely not realistic. But make sure, you think about it and determine how long you want and can support your API’s. Document your API’s over time. This gives you the knowledge to know when an endpoint was created and gives you a clear timeline when you can remove that “legacy crap”.
If you have the luxury of doing this in the context of a corporate environment, make sure you have statistics available on the most used versions. You definitely don’t want to get into a situation where you still need to maintain that “old legacy endpoint” when you don’t know if anybody even still uses that old legacy endpoint.
As a general rule, you have clients depending on the behaviour of your API and the information that is available in your API but your API is bound to change over time. Try to create a decent balance between supporting old clients whilst still providing enough new functionalities, not getting crazy because you still have to maintain legacy and make sure you have mechanisms in place to encourage people to update to the latest possible version.
It’s absolutely a necessity to have automated testing and error reporting
Running a project like this, makes it absolutely necessary to have automated testing in place (unit, feature and/or e2e). What you don’t ever want is a situation where the client (in this case an mobile app) crashes because the backend returns an unexpected HTTP 500 (or 404) with a stack trace or an “Unexpected Exception” pop up.
Testing makes sure you have the framework in place, so that when stuff changes (a refactor or even something as simple as a dependency update) you have a way of verifying that the backend still functions as it should be.
And once sh*t does hit the proverbial fan, make sure you have tools available like Sentry or Bugsnag and that they provide you with value. Besides the actually exception, having context of the exception in Sentry or Bugsnag is also really helpfull. In the case of a mobile app talking to an API, think off app version(s), any support libraries or app dependencies versions and any other details which are invaluable in the debugging.
Don’t assume your API is always available
Talking about when sh*t hits the proverbial fan, regardless if you use Amazon WebServices, Google Cloud Platform, Digital Ocean, Scaleway or some other service. Always take into consideration that your API can go down. It’s not a matter of “if”, it’s a matter of “when”.
Have measurements in place to monitor your servers and/or API. You can use something as simple as Pingdom as just monitor the / route and my some extra routes. But you also setup a complex monitoring installation with Zabbix, where you monitor (some or all) endpoints, keep historical response times and make alerts based on those response times.
Should you go with a more complex monitoring setup with Zabbix, you can always also use that same data to see if any code changes had an unexpected performance impact and could help you debug those issues.
What did I learn from it, from a non-tech viewpoint
Not all your users want the same thing. In the context of Ludum, some users might directly want to see the prices of a game at different stores or ecommerce sites. Whilst others would just like to see the reviews of that same game. Context is king in such instances and ideal case scenario, you allow your users to configure that in your app.
From a design perspective, it wasn’t the most elegant or most polished design ever.
So that meant, that I couldn’t assume the new user would automatically grasp what the app was capable off. So I needed to implement an “first run tutorial” which explains to the user what the app could do and what button does what.
If you want to break-even or heck even make a profit. Think of ways on which you can monetize your application. In my case, it was a combination of non-obtrusive ads (only a small banner ad on the bottom of the starting page) and affiliate marketing. It was never meant to be generate a boatload of money, but get my server costs covered.
Where did it fail
I want to say, it failed on the marketing part. But there was more to that part of the story. Yes I was unable to attract any real amount of users and maybe because there wasn’t a userbase that wanted an app like this in the first place.
To be honest, I don’t know. I never did any actual market research besides the asking a few friends and thinking to myself, “I would use this”. Since the idea came to me while being in the local “big box retailer” and browsing through the games and fiddling around on websites with my phone because I wanted to know if a particular game was “good” and decently priced.
But there was so much else broken in the app itself.
-
I was missing allot of data on games.
There was allot of data on games missing. It really looked bad, when somebody opened up the app for the first time, tried to scan something and saw the “Not found screen”. Even though, the data was being fetched on the background on the API and that was communicated back towards the user. That really doesn’t matter anymore, my app was supposed to solve an problem they had (lack of knowledge) and my app was only saying to them “Sorry, I don’t have that information”. That’s not a good user experience, especially if they are a first time user.
-
Incorrect data
Correct data was was far and few between. Allot of database entries were (initially) garballed up, games had wrong platforms attached to them. Prices were sometimes wildy incorrect (€ 60.000,- is one which comes to mind), images were missing. That problem, kept on dragging on and on.
The question I keep asking myself, was initially going “mobile-only” the best way ? Wouldn’t there be better ways of attracting users? So I started building a web version, which would also expose the same data. That also had the added benefit of Google scraping and indexing my data.
Planned features which never came to be
There were allot of missing features, which were on my “roadmap” but never came to fruition.
One of the biggest features, that I planned was a recommendation engine. To provide users with an revelevant recommendation based on what they were currently looking at. So for example, so them the recommendation of “Warcraft 3” when they were already looking at a different Real Time Strategy game.
To reflect back on the “Incorrect data” part of the, I initially had planned to have a “feedback” button in the app. That could then be used to send in corrections for a game.
Conclusion
Writting this post-mortem is the result of a Twitter post of mine and is supposed to answer some questions, but unfortuntaly it only raises more questions then it answers.
Serious question time Twitter!
— Saas (Sander as a Service) (@Hertog6) May 13, 2020
A while ago, I made the (painful) decision to decommission a product I have been working on for the last 3,5 / 4 years. Would anybody be interested in reading a blogpost about it the reason(s) why and technical lessons learned ?
But the legacy that Ludum left me with is that, there is so much more then development on a project like this. In a project like this, you are wearing a lot of different hats and all of them at the same time. Besides being the developer you are also the product owner, marketing person, doing infrastructure etc etc. And depending if you want to break-even, doing some (basic) accounting and deciding what the best route for monetization is, to name a few things.