Journey Platform: Low-code notification workflow platform that enables technical and non-technical customers to create complicated workflows via a easy drag and drop person interface.
By: Arjun Raman, Ken Snyder, Mengting Li
Efficient communication hinges on delivering the appropriate message, to the appropriate viewers, on the proper time. At Airbnb, our purpose is to have interaction our customers — each friends and hosts — by delivering inspirational and informational notifications via varied channels, akin to e mail or in-app messages.
Traditionally at Airbnb, complicated notification workflows have been solely managed by engineering groups, with every workflow requiring the deployment of code. As our platform advanced, we acknowledged the necessity for a low-code or no-code resolution to streamline the creation of those intricate notification workflows. In response, the Advertising Expertise group developed the Journey Platform, a robust instrument that allows non-technical customers to construct and ship customized notifications primarily based on our customers’ engagement with Airbnb.
The targets of the Journey Platform are:
- Empower customers to simply create event-driven notification workflows utilizing an intuitive drag and drop interface.
- Allow real-time execution of those notification workflows for well timed and related communication.
- Supply a unified interface for managing transaction notifications, akin to upcoming journey reminders and promotional notifications.
- Assure Service Stage Agreements (SLAs) for processing varied forms of notification workflows, together with transactional and promotional communications.
- Scale back the time required to develop complicated notification workflows.
Journey Platform permits customers to iterate quicker by permitting self-serve workflow creation. It has diminished the time taken to help a brand new use-case from 1–2 months to only 1–2 weeks.
The important thing elements of the Journey Platform are:
- Journey Platform UI: WYSIWYG instrument permits customers to pull and drop elements and create a workflow. The workflow definition is then transformed to a customized DSL (Domain-specific language) which might be interpreted and executed by the workflow orchestrator.
- Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
- Platform Retailer:
- Occasion Retailer: Pre-configured catalog of Kafka occasions which Journey Platform can hearken to and set off new executions of a workflow or cross occasions to present workflow execution.
- Motion retailer: Repository of predefined, specific-purpose features permits customers to carry out varied duties, akin to sending emails, push notifications, or emitting Kafka occasions. Customized actions might be outlined and built-in into the instrument, making them accessible to all Journey Platform customers.
- Attribute retailer: Central repository for important knowledge, akin to person metadata (e.g. person’s geolocation, Airbnb search historical past, and many others.) and contextual info. It helps decision-making in workflow branching processes by exposing these knowledge as a parameter to set situations upon via the parameter supervisor.
- Customized shops: Means to create customized motion or attribute shops which aren’t already outlined within the platform.
- Workflow Orchestrator: Mind of the system, the workflow orchestrator takes within the workflow definition DSL from the UI. As soon as a workflow is launched, it listens for occasions from the occasion retailer that may begin the execution of a workflow, interprets then parses the DSL to execute workflows on the workflow engine, and depends on the Motion retailer to carry out particular duties.
When crafting the UI for the workflow automation system, we aimed to create a well-recognized and intuitive expertise. Drawing inspiration from movement charts, productiveness instruments with “inspector panels,” and incorporating drag and drop performance, we wished a platform the place customers might begin instantly with out consulting the handbook.
We additionally had a purpose of utilizing progressive disclosure to incrementally allow the total depth of the platform capabilities, whereas retaining it easy for customers who solely want a small subset of the options. Through the use of wise defaults, and transferring extra complicated options into tabs and sub-screens, our superior customers might create distinctive options, going past the pre-planned use circumstances.
To edit the graph, we leveraged React Flow, an open-source library. This enabled us to show the graph, in addition to present primary operations like zooming, panning, transferring, and connecting nodes. On prime of this basis, we added our customized node and edge elements, together with drag and drop performance for including new nodes and an inspector panel for enhancing present ones.
To create the varieties displayed within the inspector panel, we carried out a schema-based type system. This technique supplies a excessive degree of flexibility, permitting us to declaratively specify the UI for particular node enter/output fields as a part of their sort definitions. The system is in-built a type-safe method, making use of Thrift annotations and Java reflection. Primarily based on the schema info and UI-specific annotations, the interface shows the suitable type fields, assist textual content, and validation, making certain our UI is routinely up-to-date with the platform’s capabilities.
Area-specific language
DSL supplies a excessive diploma of flexibility and customization, permitting us to outline the construction and conduct of the workflow. As an alternative of getting to hardcode a workflow within the workflow engine, we as an alternative have a generic workflow outlined that may execute any DSL-based workflow. Nodes and edges make up a workflow, with nodes representing particular person actions or duties and edges defining the dependencies and relationships between them.
The nodes and edges embrace all the mandatory info to outline a workflow akin to inputs, outputs, and parameters handed between nodes. The DSL generated by the UI is handed to the workflow orchestrator, the place the DSL parser executes it.
Journey Shops
The occasions, attributes, and actions shops are an integral a part of the backend, as they permit listening to occasions to begin workflow executions, filter customers, and execute duties within the journey. All these elements work collectively seamlessly to create a versatile and customizable backend that may be tailor-made to the particular wants of the platform.
Occasion Retailer
Journey Platform helps listening to totally different Kafka occasions and utilizing them to set off new executions of a workflow, or use the occasion to cross indicators to a working execution. For instance, begin a brand new execution when a visitor books a keep, cross a sign to a working execution when a person receives a push notification, and many others. Just like the motion retailer, as soon as an occasion is on-boarded, all of the groups at Airbnb can use it.
Attribute Retailer
The attribute retailer features as a central repository for fetching all needed knowledge, akin to contextual knowledge, person preferences, and gadget info, which can be utilized to complement the workflow branching course of and enhance decision-making capabilities. These shops are supported by a knowledge storage system that manages varied attributes or traits of entities.
Think about you will have a brand new person who simply signed up for Airbnb, and also you’re excited by figuring out whether or not they’ve carried out any itemizing searches on the platform. If the reply is sure, you’ll ship a personalised message primarily based on their search historical past, and if it’s no, you’ll ship a static message.
It is a concrete instance of how the Airbnb Journey Platform leverages attributes, akin to “itemizing search historical past,” to boost the person expertise. These attributes are extracted and outlined as parameters, which can be utilized for varied functions. Every workflow execution has its personal parameter knowledge assortment, which might be accessed within the parameter supervisor. Extra details about parameters can be mentioned within the parameter supervisor.
Motion Retailer
The motion retailer is used to execute varied duties, akin to sending an e mail or updating a database file, when a person reaches a particular level within the journey. It’s a widespread library the place every perform might be shared and reused by totally different customers of their workflow.
Every motion implements a standard interface, together with its metadata required for the UI schema-based varieties talked about above, and its conduct in the course of the precise workflow execution.
Parameter Supervisor
Managing a fancy workflow that includes a number of steps with various inputs and outputs could be a difficult activity, particularly if the enter and output parameters change incessantly or are totally different for every person. As an example, you would possibly want conditional branching in your workflow or customized communication content material primarily based on person search. That is the place parameterized workflows and parameter managers can show to be invaluable elements.
By specifying inputs and outputs (of attribute node / occasion node / customized node) as parameters, you possibly can reuse them all through your entire workflow execution. A parameter supervisor is a crucial part that may retailer and handle your workflow parameters, streamlining the method of making, storing, retrieving, and modifying them.
Along with offering an environment friendly parameter administration system, a parameter supervisor additionally supplies a variety of options akin to parameter creation, storage, retrieval, modification, versioning, entry management, and auditing. These options make sure that your workflow is executed reliably and persistently whereas additionally correctly managing and storing your parameters all through your entire workflow.
The Workflow Orchestrator executes workflows by deciphering the which means of every DSL node and performing the corresponding actions. It manages low-level features akin to storing state, interacting with the motion retailer to carry out an motion, listening for callbacks via the occasion retailer, and permitting builders to focus on workflow logic moderately than technical particulars. Journey Platform makes use of Temporal because the underlying workflow engine for state upkeep and orchestration. Temporal helps orchestrate workflows via Temporal Workers.
Builders can incorporate customized performance akin to new nodes or edges to broaden platform capabilities, making it less complicated to create workflows that fulfill the platform’s and customers’ distinctive necessities. Moreover, it helps superior options like parallel execution and computerized retries and enhancing platform reliability and efficiency.
Making certain SLA for processing several types of workflows (i.e. transactional and promotional) is crucial at scale. Transactional notifications initiated by person motion (e.g. reserving affirmation, visitor/Host messaging, and many others.) have a strict SLA and require increased precedence when in comparison with promotional notifications. To attain this, we’ve carried out the next at totally different components of the system:
Occasion pre-processing:
- Pre-filter: As an alternative of passing all of the occasions on to the Workflow Handler, the occasion processor filters out occasions that don’t match the factors. e.g. solely cross if the occasion sort is reservation_complete and filter out for all different reservation occasions. This tremendously reduces the QPS feeding into the system.
- Combination excessive QPS occasions: Occasions like searches have a excessive QPS. As an alternative of straight processing, we batch and mixture them over a time window. This reduces the QPS by a minimum of a number of orders of magnitude.
Devoted lanes:
- We’ve devoted lanes for various classes of workflows via the system. The occasion listener has totally different client teams with built-in throttling. The workflow handler has devoted Temporal namespaces for every class and strict limits on the processing QPS, max QPS to the database, and many others.
The Journey Platform empowers non-technical and technical customers to create complicated stateful workflows via a easy drag and drop interface. By leveraging a generic workflow definition DSL, together with motion retailer, occasion retailer, and attribute retailer, the platform facilitates the creation of workflows that reply to real-time occasions, streamlining communication, and enhancing person experiences.
Enthusiastic about working at Airbnb? Take a look at these open roles.
Because of Balaji Kalaimani, Davis Wamola, Iris Feng, Jesse Garrison, John Bernardo, Kumar Arjunan, Michael Endelman, Priyank Singhal, Steve Krulewitz, Tej Sudha, Victoria Gryn, Xin Tu, and Zhentao Solar for his or her contributions in constructing Journey Platform.
Because of Sagar Naik and Michael Kinoti for his or her management and supporting us on this Journey.
All product names, logos, and types are property of their respective house owners. All firm, product and repair names used on this web site are for identification functions solely. Use of those names, logos, and types doesn’t indicate endorsement.