Implementing UI Components, Plugins System and Data Access

UI Components

These 2 weeks have been very fruitful. I’ve managed to implement most of the UI components and polish the UX a little. In the previous devlog, I’ve merely scratched the surface and built just the input and button elements. Then I spent quite a lot of time to build the datepicker and a select-from-datatable component, but it was fun and satisfying to see them come to live.

For the datepicker, I decided to build something that kind of mimic the scrolling of year/month/day list on the mobile. The most notable thing I learnt from building it is using the offset and scroll attributes, which will scroll the currently selected year/month/date to the top so that the user is aware of these values being selected.

Then for the select-from-datatable component, I built it as an extension of a simple select component, so that I can asynchronously load paginated data for selection. This is mostly used in the case of selecting association data, eg. when I need to choose a tag created in another data table to use in a transaction. What is interesting for this component is the idea of composition, where multiple simpler components are used to build a more complex component.

Plugins System

During the early phase of building the app, I already had the idea of building a plugins system, such that each UI/APIs set can be self-contained in its own JS file/directory, and then easily added to the Electron app without rebuilding the entire app. This is something useful if I were to allow other people to build their own plugins and add features to the app without my involvement.

But more importantly, I wanted to make it easy for myself when I’m building the app. It proved to be a very useful strategy once I started adding more UI and APIs. I have a simple script that creates the boilerplate code for a server-side module or a UI page. When I need to add a new model, I use the script to generate the boilerplate files, then add the model schemas and UI layout. Once I build them, I just copy the bundled JS files into the Electron app and run the app without having to repackage it.

One notably interesting concept I applied in building this system is Dependency Injection. Initially, I just simply require the server dependencies within the “plugin”. This has worked fine when I run the app in development mode locally. However, when I run it as an Electron app, this didn’t work anymore.

At first, I thought that dynamically reading the plugin index file failed, but I noticed that the API routes were registered, so file loading has no problem. Then I realised the real issue is with the plugin trying to require some server dependencies, like data access module. I think it’s because in the bundled Electron app, the directory structure no longer applies, so the plugin can’t require via relative paths. The solution is to inject the server dependencies when require the plugin module.

// server.js
async function loadPlugins(app) {
  await fsPromises.readdir(path.join(__dirname, 'server/modules'))
    .then((files) => {
      for (const file of files) {
        const pluginPath = path.join(__dirname, 'server/modules', file, 'index.js')
        const plugin = require(pluginPath)(dataAccess, routes);
        const pluginRouter = plugin.router;
        pluginRouter.routes.forEach((route) => {
          app[route.method](`${pluginRouter.prefix}${route.path}`, route.handler);
        });
        console.log(`Loaded plugin: ${plugin.name}`);
      }
    })
    .catch((err) => {
      console.error(err);
    })
}

// plugin index.js
module.exports = (dataAccess, routes) => {
  const stores = require('./stores')(dataAccess);
  const router = require('./routes')(routes, stores);

  return {
    name,
    stores,
    router
  }
}

Data Access

During the design phase of the app, I mentioned that I’ve designed the models without a specific data store framework in mind. When I started implementation, I decided that I will use plain JSON files as data store and write my own data access APIs.

There’re a few reasons for the decision.

1. I want the app to be as lean as possible. If I were to use an existing data store solution, be it SQL-based or NoSQL, I would have to install additional packages and the database into the app. By using JSON files, I can omit any additional package installation, and no need to force install a database on the user’s machine.

2. I can easily reflect the data schemas, by reading from a custom schemas structure, instead of reading the schemas from the database and interpret it accordingly. Maybe there’s a package that can do what ActiveRecord does, but that will mean yet another dependency. Instead, I just define my own schemas structure, and use it for building data validation etc.

3. The data are comprehensible by the user. Since these are just JSON files, the user can easily manipulate them further to import to other apps when needed. Perhaps they started with using this app, and as their needs grow, they want to migrate to another app. No doubt the data won’t be 100% fit for the other app, but at least it is easy enough to manipulate even for a non-technical person using JSON editors.

4. The app can then technically support cross-device access. I intend to allow the user to choose the location of the data files later down the road. For example, if the user puts the files on iCloud, and configures to load from there, then technically she can access the data across her MacBook and iMac.

Of course this comes with its own disadvantages.

1. I have to write my own data access APIs. There’s no SELECT * FROM TABLE query language to use, I have to write my own version. Luckily it’s just JSON data, so it’s not that hard to implement the query logic.

2. I have to create my own indexes. Yes, like unique indexes and foreign indexes. I have to design my own index structure, which is also another JSON data file, so that I can enforce unique and foreign constraints.

3. It might be slow. Commercial databases have been in existence for a very long time, with battle-tested performance. If the data grow too large, then JSON access is definitely going to be quite slow. But I think my app won’t get to that kind of scale for me to worry about optimisation at this point, when it happens, I will deal with it again.

{
  "transactions": {
    "fields": {
      "id": { "type": "text" },
      "type": {
        "type": "enum",
        "enums": {
          "income": "Income",
          "expense": "Expense",
          "incomeReversal": "Income Reversal",
          "expenseReversal": "Expense Reversal"
        }
      },
      "transactionDate": { "type": "date" },
      "description": { "type": "text" },
      "amount": { "type": "number" },
      "tags": { "type": "array" },
      "currencyId": { "type": "text" },
      "associatedTransactionId": { "type": "text" }
    },
    "constraints": {
      "foreign":{
        "tags": { "reference": "tags" },
        "currencyId": { "reference": "currencies" },
        "associatedTransactionId": { "reference": "transactions" }
      },
      "unique": [],
      "required": [
        "type", "transactionDate", "description",
        "amount", "tags", "currencyId"
      ]
    }
  }
}

That’s All For Now

That’s the result after 2 weeks of work, I think I’ve made quite a significant progress in setting up the foundation of the app. Next, I will focus on building the business logic and hopefully not have to digress too much to polishing the UI components.

There’s a demo page for the UI components I’ve built so far, you can have a look at it if interested.

Stay tuned for more updates!