Published on
Reading Time

Building a JS DSL

Table of Contents

Overview

Building a DSL to bridge between multiple view libraries in UI layers.

Disclaimers

I was the first individual contributor hired on the UI team for Treasury Management at Jack Henry.

That team grew, mainly because of the following 2 individuals:

This is a summary of problems we ran into, why we solved that way, and reflections on what Id change.

Lots of things went well, some didnt.

I was asked to talk about this, and rather than writing notes once, I figured Id share the notes publicly.

Problem

Jack Henry Treasury Management was originally built on Angular.js. The product evolved rapidly and new features were added daily.

The UI on Angular.js was great in 2015. Bad in 2019.

We needed to get off of Angular.js ASAP.

  • Feature updates stopped in 2018. Attracting talent interested in working on an Angular.js app in 2020/2021 is really hard.
  • Security patches were ending in Jan 2021 (extended because of Covid). We sold banking software. This is self explanatory.
  • $scope led to shared state (and public state) any view/controller could override / mess with. We want each "rule" to own its own state, and be the only arbiter of changes to it.
  • Performance - The digest cycle in Angular was a black box that caused performance issues with little ability to understand why. The complexity of our UI often led to slow experiences for our users.

The below highlights decisions and thoughts about this problem.

Solutions

  • Rewrite into another front end library (React, Lit, Vue, Angular, etc)
  • Ensure modularity with the "core" business logic. This would let "sub teams" ensure their "module" worked without having to understanding all the other pieces. I became the ACH expert. My day to day was solving ACH problems, and other team members were able to focus on International Wires or Bill Pay.

Trade offs

  • Rewrite into another front end library (easier) - leave logic in the view

    • This model is the status quo for front end teams.
    • Easier to hire
    • Expertise exists
    • Leadership has previous experience doing this (easier to understand timelines)
    • Certain frameworks provide "batteries" out of box, which might make other problems in UI easier
    • Lets developers work on the "new" thing (upskill)
  • Move logic out of the view library (harder)

    • This model is unproven, and unclear what this means
    • Not easy to hire
    • Expertise is more expensive (hand in hand with harder to hire)
    • Requires Leadership to take a big bet
    • "Roll your own" can lead to sunk costs when things arent working
    • Harder for devs to communicate what they know
    • Requires a strong product vision, and enforcement of that in (general) business rules
    • Gives a needed technical challenge to great engineers

Decision

  • Build a DSL to move all business logic out of view libraries, and into plain JS. This was harder, but would be the last time Treasury Management had to do such a large refactor

API Design

  • Reduce dependencies
  • Fluent API
  • All methods are chainable
  • Open for extension, closed for modification
  • Readable by Product people

These were our guiding principles in creating the DSL (later named Omega).

Business logic associated with moving money online for businesses is incredibly complex.

We were not dealing with SMB companies with 3 employees, we were dealing with fortune 500s with very complex approval processes.

A few examples of this:

  • Claims(Permissions) - based on what type of money movement, configurable by bank, company and user.
  • Same Day ACH (SDA) Limits - based on cut off times at bank and company level (and NACHA/FED requrements). If amount >$25,000 SDA is not an option.
  • Effective Date - based on cut off times, product type, bank configuration, future date requirements (based on payment type)

Reducing Dependencies

  • We worked hard to ensure each "module" had minimal external dependencies.
  • Less direct dependencies and more "events". aka Observable patterns.
  • Fire an event and let any one who cares about that subscribe to act on it. This decouples any direct dependency from the sender and receiver of that event.
  • Any module who cares about that event could act on it.

Open for extension, closed for modification

  • This logic is incredibly complex. Devs should feel confident making changes needed without risk of breaking 100 other implementations.

Chainable

  • Most devs have used a chainable API before (for example, JQuery).

Readable by Product people

  • Given these business rules are so complex, looking at the logic and being able to read it (without understanding all the underlying details) during quick standup meetings would help to ensure we are all on the same page.

What might this look like?

const sameDayAch = new FieldType().thatIs
.disabledWhen(amountGreaterThan25k)
.and.disabledWhen(pastFedCutOff)
.and.required()
.with.formatter(moneyFormatter)
  • Product people could read this, and help us understand we are missing another condition. SDA should be disabled if the user does not have the SDA claim.
const sameDayAch = new FieldType().thatIs
.disabledWhen(amountGreaterThan25k)
.and.disabledWhen(pastFedCutOff)
.and.disabledWhen(userIsMissingSDAClaim)
.and.required()
.with.formatter(moneyFormatter)

Ok...that...looks cool. How does it work?

Architecture

  • Return a new copy of the "class" / "object" each time a method is "chained".
  • "Public" methods modify the behavior. "Private" properties internally control if something is disabled.
  • isDisabled is the public method. the disabled Property is used internally (and passed down to the actual components to be disabled)
  • Small Single API - There is only one way to implement the behavior. This makes it easier to prove our logic works.
  • Pass a predicate function into any chainable method to control true/false
  • Helper methods (.required() does same thing as .requiredWhen(truePredicate))
  • Mapping "component" called omega-field uses this sameDayAch object, extracts the "private" properties, and passes them into the individual components in the correct way.
<omega-field-angularjs .field="${sameDayAch}"></omega-field-angularjs>
<omega-field-lit .field="${sameDayAch}"></omega-field-lit>
<omega-field-react field={sameDayAch}></omega-field-react>

*** Note - Markdown is preventing me from inputting the correct field=${sameDayAch} syntax above. There should be backticks around the variable, but markdown doesnt want us to have nice things.

  • The special field components extract this "magic" field passed to them, and return the actual right component.
  • Key distinction - the special field component should be the only component that knows about this special field.
  • All other components should just be given their own disabled prop passed from omega-field.
export default class OmegaFieldLit {
onChange(){
/*
logic to pull all the properties out from the "sameDayAch" object and put
in named properties like .options, .disabled, .required
*/
}
render() {
if (this.field.hasOptions) {
return <omega-select
.options=${this.options}
.disabled=${this.disabled}
.required=${this.required}
.multiple=${this.multipleOptions}
>
</omega-select>
}
}
}
customElements.define("omega-field-lit", OmegaFieldLit)
  • We tried to keep all of the "Core" Logic in Vanilla JS. And framework/View logic held in a field component like above.

Why Vanilla JS

  • Unit tests > E2E Tests using a framework.
  • Modifying behavior can be changed with a single new function.
  • Plain JS runs everywhere (webpack/polyfills are assumed here!)
  • Forces devs to get really good at writing logic! (no framework to cause side effects)

What Id change looking back

  • Typescript wasnt as popular in 2018/2019 when this project got off the ground, but today, Id use that
  • Smaller scope - This went well, which led to an ever increasing "we can also tackle this problem too".
  • Tackling that problem too led to the team being spread thin, and lots of internal custom logic.

Example of extending

const sameDayAch = regularAchPayment.thatIs
.disabledWhen(amountGreaterThan25k)
.and.disabledWhen(pastFedCutOff)
.and.disabledWhen(userIsMissingSDAClaim)
.and.required()
.with.formatter(moneyFormatter)
.with.options(getPossibleRecipients)
  • The additional .options took an async "fetch" function to grab data from an API. The "side effect" here was .options also caused the field to swap from an "input" to a "single select". The above example shows a bit of how that would work.

  • We ran into issues with this, and eventually had to add the ability to pass an object with some mapping functions in. (when the value you wanted to display in the select didnt line up with the underlying value you needed to send to API)

```javascript
const sameDayAch = new FieldType().thatIs
.disabledWhen(amountGreaterThan25k)
.and.disabledWhen(pastFedCutOff)
.and.disabledWhen(userIsMissingSDAClaim)
.and.required()
.with.formatter(moneyFormatter)
.with.options(
{
data: () => getDataFromAPI,
text: (record) => record.valueToShowOnScreen,
value: (record) => record.valueToStoreInternally
}
)
  • You could chain .multipleOptions() on to enable multiple select behavior.
const sameDayAch = new FieldType().thatIs
.disabledWhen(amountGreaterThan25k)
.and.disabledWhen(pastFedCutOff)
.and.disabledWhen(userIsMissingSDAClaim)
.and.required()
.with.formatter(moneyFormatter)
.with.options(getPossibleRecipients)
.and.multipleOptions()
  • More abstractions led to longer ramp up times, key man risk (lots of functional expertise to be comfortable with this) and complexity when the "business" rule didnt quite follow in this use case.

What was successful?

  • This did ultimately lead to clear separation of business logic from presentation logic.
  • Views imported the "business logic" needed for a "form" or "field" and Omega would run each function, do change detection, re-run, and update itself.
  • Massive upskill in team. This sort of "abstraction" led down a functional programming experience most had never worked with before.
  • Challenge - At a point, CRUD is boring. This presented a hard challenge for a very skilled team to wrestle with, while shipping value to end users.
  • Legacy code and "new" view code in Lit used same underlying logic.
  • Side by side development, allowed easy MVPs and fast iteration.
  • Empowered UI team to expirement

What was unsuccessful?

  • This led to incredibly hard to solve bugs.
  • A particular function would be passed around 10-20 times, and following exactly where you lost the context, or this was pointing to the wrong object led to many gray hairs.
  • At the time I left, there were just as many unfinished features as finished.
  • You could objectively look at this, and say it was a distraction to shipping contractual commitments to customers.
  • Required dual, or even triple, expertise. Angular.js, extremely functional JS, and Lit

Learnings

  • Got very comfortable with functional JS
  • Value of hard engineering problems
  • Value of product management for the library
  • Do it again, with smaller scope.

Credit