Slate v4 Development

Documentation

A large push for Slate v4 is to document as much as possible. Before developing new modules, or making functionality changes, we're pushing to document what is needed, and how we're going to implement it. Getting sign off from the stakeholders will help reduce major changes late in the development of the modules. It also will provide us with records of just why a module ended up with odd functionality.

Backend Technical Debt

The backend of Slate has some dusty corners. Many of the core modules were written nearly a decade ago. Better code patterns have been developed. Library features previously unused have been refined. Often these updates and cleanups are deprioritized in favor of pressing business needs.

Part of our goal for Slate v4 is cleaning up our backend.

Overall tasks for the Slate v4 backend

• Create generic views inheriting from the DRF generic views.
• Create generic filters for user in generic views
• Create pagination that is compatible with existing frontend

Per module tasks

• Switch to Slate generic views
• Create unique filters
• Check that permissions make sense
• Integration Testing
• Revise code style (e.g. decorators vs properties assigned to functions)
• Apply auto-formatting for code quality
• Review if API 'shape' makes sense.
• Review models for database optimization (indexes, column restrictions, etc)

Porting the Frontend

AngularJS, the framework Slate's frontend was developed with, has been end of life'ed. While the framework still works, the support for it is aging. The in browser devtools for AngularJS will cease to function in early 2023. The node version the build tooling will run on is also capped at node 12. In addition, only a specific combination of versions will build on node 12.

The successor we've chosen is VueJS. One of the major reasons Vue was chosen is it's similarity to AngularJS. There are many other reasons for using Vue, but speed of porting for us will be key.

Currently we have a proof of concept for porint. We used the seperate vendor site as a test bed for the port. We've implemented our API access as classes that work similar to the AngularJS services we currently use. Class inheritence significantly speeds up development. Most of our API access follows common patterns, and not maintaing seperate implementations helps immensely.

New design elements have also been tested. The feasibility of new navigation patterns were also tested with the vendor site. We've refined the concept of a sidebar/actionbar interface for Slate v4. An exploration of design has been completed by on of our internal designers. This will be developed into a custom bootstrap theme and style guide.

Currently we have a repository created for what will be the Slate v4 frontend. Build tooling and supporting packages have all been setup. For obvious reasons authentication is the first module under development, soon to be followed by core modules as they go through code cleanups.

Project Plan

Stage One

One of the major intentions of stage one is to not cause disruption to the current development of slate. This stage can run along side with other work in the current version of slate. Including small amounts feature work, and general bug squashing.

Setting Code Standards

Stage one will focus on setting code standards for the Slate backend. We have developed typical patterns we can expect a given module to use. These patterns are old and we have not reviewed them in years. We plan on targeting new coding standards and conventions with the following;

• Better use of DRF class views
• Use of backend filters vs duplicating logic
• Better Pagination using Deferred Joins
• Fixture generation for testing / development scenerios
• Automated testing
• End to End testing for api functionality
• Unit testing for utility functions where appropriate
• API Documentation

Once we have updated one module, we can adjust the standards for practicality. Once we have refined the standards, we can start updating the simpler modules. We don't expect breaking changes due to updates of the simpler modules. Where modules need changes, required changes should be negligible. Our initial target modules will include;

• Billings
• Bom
• Companies
• Contacts
• Conditions
• Discussions
• Estimates
• Helpdesk
• Payables
• Sample Tracking
• Terms
• Time Tracking

These modules are not expected to require any custom code. This means they can take advantage of the standard class views. They may require creating fixture generation in the modules they depend on.

Task Queue Swap

We currently use Celery as our task queue. Celery is a far heavier and more complicated task queue than we require. This has lead to issues with memory usage, worker scaling, and lockups. We need to proceed with the planned switch out before we break for stage two. We've switched away from RabbitMQ as a broker. This was due to scaling issues, as well as being too much software for our needs. We currently use Redis for our broker, and have not seen issues due to that switch.

We're planning to switch to a task queue called Huey. It's limited to Redis as a broker. However, we're already using Redis as a broker, so this won't result in a change.

The actual changes will result in very few code changes. Most of the tasks we run are one to two lines. This means scheduling and task setup are our only changes, with minimal code touched.

Okta Integration

We currently have proof of concept code working for logging in using okta. With Berlins heavy reliance on okta it makes sense that this should be functional before we move onto v4.

Currently the user would already need an active account in Slate to login using Okta. Okta would only be acting as an identity provider at this point.

Stage Two

Stage two represents the point where development is split between v3 and v4. We intend to continue maintenance on v3 while v4 is under development.

Below are the major tasks and updates planned. They not neccesserily in chronological order. This list is also subject to additions.

De-Jansyfication

Many places in the code, we have the fact that Jansy is user of the software hardcoded. Fields will need to renamed, and anywhere that copy mentions Jansy will need to be rewritten

Domain move to slate-erp.app/slate-erp.com

Due to Reaction Retails situation, we've had to hide any Jansy ownership of slate that the RR clients might see. So we have bought the slate-erp.com and slate-erp.app domains. V4 represents an opportune time to switch our usage to these domains. This will silo slate away from anything the marketing teams might want to do with jansypkg.com domains.

slate-erp.com is intended to be used for a basic landing site, as well as any documentation and training materials on an appropriate subdomain.

slate-erp.app is intended to be used for instances of Slate itself. Typically these instaces would live at jansy.slate-erp.app or reactionretail.slate-erp.app as they do now.

Productiziation

While Slate is, and will remain, a unified application, many sections work well as discrete units. For instance, there's the CRM part, the Project Management part etc. We want to break down our permissions setup, and access, into these blocks. Being able to turn 'Products' on and off on a per user or per group basis would be helpful. There currently exists 'MODULENAME_can_access' permissions, but they're actually too granular to be useful. Often what we want is not to turn off a module but to subset the data available.

Continue Code cleanup

As per the preceeding section, we'll continue cleaning up each module. In this stage we'll also be targeting the modules with custom business logic beyond the typical CRUD methods. Most of this work will be moving functionality into the models and serializers, and out of the views.

UI / design stuff

Part of our upgrade to Vue3, we're also applying a fresh coat of paint to the UI. Initial, exploritory, designs have been done, and can be viewed here.

Some time will need to be put into developing the custom UI components we use. Most of the heavy lifting will be done by existing UI frameworks and libraries. However we do have some components which are combinations of base components that we use, especially number inputs and currency handling.

Git Branching and Pull Requests

Currently our source control has a main branch, and a Production branch. We've also had issues with tracking work-in-progress due to long lived branches residing on a developers computer. This can lead to conflicts between these branches and the main branch, as well as losing transparency of what's being accomplished.

Going forward with v4, the main branch should be deployable at any time. Work-in-progress should continue to be done in feature branches. When they need to be tested, the 'Beta' site can checkout that branch directly.

Feature branches should be made with the intetion of doing chunks of work that will take less than a week. For more complex features, these should be done behind feature switches. This way merging a branch will not break production. This should help consideration of the effects a change might have on the rest of the sysem as a side effect.

If a feature is invasive enough that it can't live bedhind a feature switch, it should still be pushed to github, and continually rebases against the main branch. Every effort should still be made to reduce the size of work in a branch to management chunks.

Porting Projects

The project module needs a cleanup. Since it has grown organically, there are areas that have been repurposed or models whose use no longer reflect their name. This is also a more complext module as it includes the design requests (project task tracking).

Some major improvements planned include inter-request dependancies and gantt charting of those requests. We also need to track lifetime project development value. Giving sales and management a look at total expenses including design time.

This is also a major update due to all the related items displayed in tabs on the project page.

Order Plan Removal

The order plan module is an independant module wich attempted to track where in the process an Order/Opportunity is in its lifecycle.

Unfortunately its been found to be too complex for general use. It also reaches it's limit or practicality with some of our more complex projects.

The order plan will need to be replaced with individual objects tracking their own dependancies.

For many projects genearting a gantt chart for Orders/Opportunities will be far more useful for users. Gantt charts are also the expected display for these kinds of task lists.

File areas

There are many pending features for file areas that we plan on finally implementing:

• Moving file nodes
• Copying/Linking file nodes
• Full featureset for widget view
• Directory Zip Download in full tab view
• File upload chunking

File upload chunking is a feature required to use cloudflare. Currently cloudflare has a 100mb body size limit on request bodies. Our file uploads can and do exceed this regularly. Most object storage backends we would use actually support storing files peice by peice due to these types of limitations. When implementation is complete we can place Slate behind cloudflare and take advantage of their feature set. Particularly Argo routing, which should speed access, particularly for our China offices, is of interest. The application firewall is also of interest to us.

Dashboard

The dashboard system will need a rewrite for the frontend. The current UI library we use will not work, and with newer web technologies, more efficient code can be used.

Customization

Along with the dashboard there are areas of Slate with per user customization. Currently these are stored on the users Contact object. This also needlessly triggers an update of the users Contact to salesforce.

We plan to move this to its own object.

Global Search

Currently the global, or top nav search, is calling multiple endpoints to do it's search. There is no fuzzy matching of objects, and any items searched for have to be implemented in the nav bar code.

We're looking to setup an actual search engine. We'll be updating it with 'documents' and letting more appropriate software do the search. This will replace our naive search functionality.

The specific search engine we're planning on using is: Meilisearch

For the implementation, we'll need to write serializers to get our db objects into the search engine as 'documents'. Followed of course by a frontend view to do the actual searching.