Opentrons Protocol Designer Update: Bulk Editing Overview

Opentrons is pleased to release improved batch editing options for the Protocol Designer. Here's everything you need to know about how it works.

This feature upgrade comes directly from customer feedback. Thank you! We'd love to hear your ideas - please keep them coming!

When Protocol Designer (PD) was first created, Opentrons envisioned a platform where scientists could quickly build basic pipetting protocols without writing any code. As PD is increasingly utilized by our community, it becomes a tool that enables scientists to build lengthy and complex protocols to address applications such as DNA extraction, protein purification, and COVID-19 diagnostics. When we started seeing scientists using PD to build protocols with over 100 steps, we knew we needed to give them better ways to optimize and edit their protocols. Scientists have enough to worry about: we don't want them to spend their time tediously changing the same settings over and over again.

So, we just added a bulk edit mode to PD! Users can now select, copy, delete and edit multiple steps, enabling faster iteration and optimization of scientific protocols.

Before the product development team even started thinking about how to accomplish this feature, we knew it would be a huge engineering challenge. Extending an application like PD to support completely different user flows (such as batch editing) would require significant refactoring, which would impact core parts of Protocol Designer. This article will focus on this feature from an engineering perspective, but I also want to point out the significant design challenges involved in accomplishing this feature. One of the core principles of PD is ease of use, and creating an easy-to-use, intuitive user experience for building and optimizing complex scientific protocols is no small feat.

Designed for editing one step at a time

PD is designed to build scientific protocols from scratch, sequentially creating and editing steps. The process is as follows:

1. User creates a step (transfer, mix, heat, etc.)
2. Fill in the form with relevant step information (pipettes, labware, liquid volumes, etc.)
3. Save form
4. Create next step
5. etc.

This means that at any given time, the main Design tab in PD will display some/all of the following:

1. List of all steps in the protocol
2. Visual snapshots of specific steps of the protocol
3. A form for the user to fill out to specify what the step should do

PD's UI components (built into React) and global state management storage (implemented in Redux) are built to accommodate this user flow. This means that PD's redux store keeps track of things like "selected steps" so that users can visually see what is actually happening at a specific part of the protocol. It also tracks things like "unsaved forms", keeping track of pending changes before submitting them.

Note the singularity of the data captured above (selected steps, unsaved form). PD's redux store and React components are designed to interact with one step, not multiple steps. To illustrate this, let's see how PD fills the holes on the labware based on the steps chosen (this is how the holes appear green in the image above).

1. User clicks on a step
2. Schedule a redux operation named "SELECT_STEP" to update the "selected items" reducer
3. A redux selector listening for "Selected Item" changes will search for labware and wells being used in that specific step
4. The hole information is passed into a component called "LabwareRender" which will then use that information to color the holes

But what happens when we start tracking multiple selected steps instead of just one? In particular, what happens in step 3 above? What should we do if we select multiple projects?

To prevent component corruption when selecting/editing multiple steps simultaneously, we had to make core changes to how data is represented and transformed. Let's review what we did.

Track multiple selected steps

Prior to this feature, we only tracked a single "selectedItem" in the Redux store. Our "selectedItem" reducer type interface looks like the code snippet below. Please note that all code snippets in this article are written in JavaScript and typed using flow js.

The reducer function "selectedItem" accepts a state and an action and returns the selected item (an object holding information about the selected item), or null if there is no selected item.

To avoid adding a new reducer to save multiple steps in bulk edit mode, we chose to modify the "selectedItem" reducer to accommodate returning single steps and multiple steps.

The return type of "selectedItem" has been modified to be able to hold an object containing a single step id (representing a single selection type), or an object containing a list of multiple step ids (representing a multiple selection type). To tell redux that we have selected multiple steps we create an operation called "SELECT_MULTIPLE_STEPS" which the "selectedItem" reducer function will accept and update its value to represent multiple steps (see type "MultipleSelectedItem")

The reducer named "selectedItem" was definitely a bit awkward, potentially holding data representing multiple items, but we ultimately decided that the trade-off was worth it to not have to add additional reducers to represent multiple selected items, It is thus necessary to cancel one or more items while switching between single editing mode and batch editing mode.

To prevent components that used to only accept a step as a prop from breaking, we were able to leverage the redux selector pattern to convert the data from the reducer into a format our component could accept. The main selector that provides information about the selected step to our component is called "getSelectedStepId" and it used to do the following:

This is a simplification of what selectors used to do, but you get the idea - it basically goes into the "selectedStep" reducer and returns whatever is in it. Because our component gets the selected step from the selector instead of the reducer, we are able to first transform the data held in the reducer before feeding it into our component.

This means that all we have to do is modify "getSelectedStepId" to return the step id when the reducer saves the "single selection type", otherwise it returns null:

Since our existing component is now able to handle multiple steps of selection, we added a new selector called "getMultiSelectItemIds" which is similar to "getSelectedStepId" but returns a list of step IDs in bulk edit mode and otherwise returns null. This selector will be used to tell PD which steps are selected in bulk edit mode.

Getting the data flow from reducer => selector => component really helps us because we are able to change the structure of the reducer without having to worry about component corruption. Additionally, since we use reselect to group the selectors together, all higher-order selectors using "getSelectedStepId" will still work fine.

Fill the bulk edit form

PD determines which fields in multiple steps are editable based on a matrix of rules. For example, if a user selects two transfer steps and the two steps have different pipettes, they should not be able to modify the pipette flow settings shared between the two steps.

Using the rules matrix, we created another redux selector called "getMultiSelectDisabledFields" which, as the name suggests, determines which fields should be disabled in multi-select mode. It iterates over all fields in the selected form and determines whether the forms share the same pipettes, labware, etc. Based on the rules for each field, it will return a map of which fields are disabled and why each field is disabled. Field is disabled. The bulk edit form component can then use this information to populate which fields are editable and which fields are not.

Track bulk edit changes

After you populate the fields of your bulk edit form, you need to track changes to their values ​​when users modify them. This is for several reasons:

1. We need to know if the user has made any changes to the form, because if they have made changes we want to remind them that if they try to exit the form they will lose those changes.
2. Once they complete their changes, we need to merge their unsaved changes (which affect multiple steps) into the saved step map within the PD global state.

For single edit mode there is another reducer called "unsavedForm" which saves all the information in a single unsaved form, but we decided not to reuse it in bulk edit mode because:

1. Batch edit forms to save information of multiple forms, not just one form

2. In bulk edit mode, it's much more useful to just save information about which form fields have changed. This way, when the user saves the bulk edit form, all we have to do is extend the changes to each affected step in the "savedStepForms" reducer that saves all saved form information. This also means that as long as the object representing the change is not null, we know the user has made a change.

To achieve this, we created a new reducer called "batchEditFormChanges" which contains a pure JavaScript object that represents the edited field name as a key, and the associated field value.

Reuse form components

PD's form components are very "smart" in that they are connected to redux and therefore have access to the form data. The problem is that the logic in the "smart" component is directly related to the single editing mode. To remove the dependency on single edit mode, we decided to inject form components in single edit mode and batch edit mode with a set of props that share a common API called "FieldProps"

To achieve this, we created two separate functions (one for single edit mode and one for batch edit mode) that are responsible for calculating each "FieldProps" above. They are aptly named "makeSingleEditFieldProps" and "makeBatchEditFieldProps". The primary parent component of a single edit form uses the former, while the corresponding component of a bulk edit form uses the latter.

Both pure functions get the corresponding single edit/batch edit status information (such as what information is included in each form), perform the necessary logic, and return an object containing the same "FieldProps" interface as above. This means that as long as all our form components accept the "FieldProps" interface, they can be used in both single edit mode and batch edit mode.

Migrating our existing form components away from coupled single edit mode logic was quite a bit of work (and we have more work to do), but creating this common props interface allows us to reuse existing forms Field component while drawing a clear boundary between single edit mode logic and bulk edit mode logic.

Main points

All of the above (and more) took our team of three engineers, a designer, a product manager, and a QA engineer about three months to complete. You can check out our epics for selecting multiple steps here, as well as our epics for batch editing form-specific work. All of our work is open source, so feel free to browse our code base.

Before we started developing this feature, we asked ourselves whether the entire quarter's investment was worth it. Since bulk editing is by far the most common feature request from users, this is what we ultimately decided to do. However, it is worth noting that the reason why users want to edit multiple steps simultaneously is because their protocols often contain many steps that are easier to edit in bulk rather than one at a time. As more scientists use PD to solve increasingly complex problems, problems with protocols containing many steps will increase.

Going forward, we hope to better understand what exactly users are building with PD and what we can do to help them minimize the steps to create. We will work to answer these questions as PD continues to evolve and evolve. While we're very excited about this feature, our mission to empower scientists to act faster and solve the problems we urgently need them to solve is far from over.