Marble diagrams for testing RxJs operators

Adrián Martínez, Software Engineer at EDICOM, explains how we can test RxJs Observables using Marble diagrams. Discover a new approach to verifying the behaviour of reactive scheduling through a graphical representation of expected functionality.

    Written by:

    Adrián Martínez

    Software Engineer

    Full-stack developer passionate about programming world. Great interest in Application Development, Material Design and Machine Learning.

Introduction

Reactive programming is one of many programming paradigms available today. It is focused on handling finite or infinite data streams asynchronously. These data are propagated through the application, making changes in the application, and our code reacts to these changes by executing a series of events asynchronously. This paradigm is closely related to the observer software design pattern.

One of the technologies we use in EDICOM for applications is Angular, a framework for developing SPAs (Single Page Applications) using HTML and TypeScript. This framework makes use of the RxJs library, which implements the observer pattern and provides a set of operators to work with data streams in a more declarative way.

In this article, we will explain what the observer pattern is, we will talk about the RxJS library and present a visual representation to understand how it works, the Marble diagrams. Next, we will show how EDICOM makes the most of this library by creating customized operators and testing them.

Observer pattern

The observer pattern is a software design pattern from the category of behavioural patterns where a 1 to n relationship is established, so that when the object (called the subject or notifier) changes, all the objects that depend on it (called subscribers) will be

updated automatically. This dependency is known as subscription.

The notifier object has a mechanism for subscribers to subscribe or unsubscribe. When an event occurs, the notifier scrolls through the list of current subscribers and executes the notification method that was specified at the time of subscription.

RxJS

RxJS (Reactive Extensions for JavaScript) is a library that implements the observer pattern for JavaScript/TypeScript and provides a set of types and operators to make working with it more user-friendly.

The really powerful thing about this library is its operators, as they allow complex asynchronous code to be used, but using a declarative programming paradigm. In this paradigm, the programmer focuses more on the ‘what’ than on the ‘how’. In other words, we focus on what we want the code to do, but without telling it the exact steps it should follow. The instructions to be carried out are already correctly programmed in the library.

As this is asynchronous reactive code, many of the operators provided by the library are time-related, so trying to explain them through a textual description may not always be sufficient. This is why the Marble diagrams came about, as we explain below.

What are Marble diagrams?

Marble diagrams are a visual representation of how the operator works. The input(s) to the operator and the expected output are presented visually, but with particular emphasis on timing, i.e. when that input occurs and when that output is delivered.

These diagrams can be represented in ASCII with the following characters:

  • ‘ ‘: Is ignored. It is used to align the different diagrams in an example.
  • ‘-‘: Represents the passage of a virtual unit of time called ‘frame’. By default, one frame equals 1ms.
  • [a-z0-9]: Alphanumeric value used to represent an output value.
  • [0-9]+[ms|s|m]: Represents the passage of a given number of frames without having to set n ‘-‘.
  • ‘|’: Symbolizes the successful completion of an observable.
  • ‘#’: Signals an error in an observable.
  • ‘()’: Synchronous group, used when multiple events occur synchronously in the same frame. The frame where the ‘(‘ symbol is placed is where the group will be output. Once the group is output, as many frames pass as the length of the group. In other words, when the group “(abc)” is output, 5 frames will pass until the next character is processed. This is done to help align the different diagrams vertically.
  • ‘^’: Indicates the point at which the observable to be tested is subscribed to the observable. This symbol is only used in ‘hot’ observables, a concept to be explained below.
  • ‘!’: Point at which the subscription is cancelled.

However, for aesthetic reasons the ASCII representation is often not used, employing illustrations based on the ASCII representation instead.

Marble diagrams for testing RxJs operators

Hot and cold observables
Depending on how the data emitted by an observable are produced, it can be catalogued as:

  • Cold: when the producer is created and activated at the time of subscription. For example, an http call in Angular is made via observables so that you can react when you get a response, but it is not until the time of subscription that the request itself is made.
  • Hot: those observables whose associated producer is created and initialized outside the subscription. By doing so, the same value can be output to multiple observers. This is achieved by using a Subject to multicast the generated value. This would be for example the value of a form; when this value changes, all your subscribers are notified instantly and with the same value.

Customized operators and how to test them

One very powerful aspect of the RxJs library is that it allows you to extend the operators provided with those needed for more specific use cases. This way, we can create an additional operator and use it in different parts of our applications whenever we want to solve the same problem, which also improves the readability of our code. In addition, as it is a new RxJs operator, we can use the RxJs library itself for testing thanks to the Marble diagrams.

To create a custom operator, we will use the ‘pipe’ static method inputting to it the series of actions performed by the operator.

Once we have created the operator, we will use the TestScheduler of the rxjs/testing package (link: https://rxjs.dev/api/testing) to validate the behaviour of the new operator by means of a diagram. This lets us run RxJ operators in a context where the scheduler in use is replaced by one that simulates the passage of time with what is called ‘virtual time’. It also provides us with a series of helper functions to write the tests.

The functions provided are:

  • Cold and hot: To create different types of observables.
  • expectObservable: Schedules the checking that the observable indicated is equal to the Marble diagram provided.
  • expectSubscriptions: Similar, but to check when a subscription occurs or is cancelled.
  • flush: Forces the start of ‘virtual time’. If not called, it is invoked automatically.
  • time: Creates an observable that only simulates the indicated passage of time.

The values output in the diagrams indicated by [a-z0-9]. However, it is often necessary to indicate a mapping between these identifiers and the actual value to be used for the test. This will be discussed in greater detail in the examples.

It should be noted that the testing scheduler is completely agnostic of the testing framework you use, so when you instantiate it you have to tell it how to validate that two objects are the same. At EDICOM, we use Jasmine and its configuration would be:

Marble diagrams for testing RxJs operators

At this point in the article, we will go on to explain one of the reactive programming use cases we have in EDICOM. First we will explain the problem to be solved, create an RxJs operator to solve it and validate its behaviour with several tests using the Marble diagram.

Use case: delayed search

In many applications we have a list of entities that the user can consult (such as invoices, messages, items, etc.) where there is also a search box for quick searches using free text entered by the user. When this text is entered, the web application makes a request to the server and will return the list of entities that meet the search parameters.

The requirements to be met are:

  • Initially all entities will be displayed.
  • When the user has stopped typing, the search will be run. This, at the typing speed of an average user, is usually when 500ms have elapsed since the last time the text was modified.
  • If the search to be made is the same as the last search run, the call will be skipped.
  • If the search box is modified while a request is in progress, it must be cancelled and the new request must be made.
  • When the call is made, visual feedback should be given that a request is in progress.
  • If an error occurs on the server, it will be assumed that there are no elements that comply with the request made.

Seen from a reactive standpoint, the problem can be explained as follows:

  • We have a dataflow; the text entered by the user.
  • The application must react to this flow by making requests to the server following the requirements explained above.
  • The requests sent to the server are themselves another data stream, connected to the first one. The application must react to them and display the results to the user.

Code

First, we create a service that simulates the call to be made to the server. Thanks to the ‘delay‘ operator, it simulates that the server takes 2s to answer.

Marble diagrams for testing RxJs operators

To program as described above, we have proceeded as follows. In the controller:

Marble diagrams for testing RxJs operators
  • We create a FormControl called ‘searchControl’ to get the user input reactively.
  • We create ‘filteredList$’ the dataflow to be displayed to the user from their input. We do this with the ‘delayedSearch’ customized operator explained below.

In the view:

Marble diagrams for testing RxJs operators
  • We have an input to which we link the ‘searchControl’.
  • We listen with the ‘async’ pipe for changes in ‘filteredList$’ to iterate over the list and display the result.
  • If the result of ‘filteredList$’ is not defined, it is because the call is in progress, in which case the user will be told that.

And finally, we create the operator to get the data flow to be shown to the user. This operator needs to be configured with the initial search value and the function to invoke to perform the search.

Marble diagrams for testing RxJs operators

This operator is created by concatenating several existing RxJ operators:

  • ‘debounceTime(500)’: Outputs the last value issued by the source when 500ms have elapsed with no new emissions. This way, we wait until the user has finished typing.
  • ‘startWith’: Serves to issue a value as soon as there is a subscription, even if the source has not yet output anything. This allows us to display the initial list without the user having to perform a search. It is important to put this operator after the ‘debounceTime’ so as not to introduce an unnecessary initial delay of 500ms.
  • ‘distinctUntilChanged’: Outputs the received value as long as it is different from the last one issued. This way, we avoid making the same call to the server twice. Note that this operator must also be placed after the ‘debounceTime’ so that the check is done with the value after the delay is applied.
  • ‘switchMap’: Thanks to this operator (formed by the operators ‘map’ and ‘switch’) we can make a mapping between the text to search for and the observable that is resolved with the list to be displayed, then it subscribes to the new operator and will output this value. In addition, if a new value arrives without the previous one having been issued, the previous subscription will be cancelled and the new one will start.

We will use this in conjunction with the ‘concat’ operator, which allows a series of data streams to be concatenated in an orderly fashion as the observables are completed.

  • ‘of(undefined)’: Observable that immediately resolves to an undefined value and is completed to make way for the next one. This value is used to know when a request to the server is initiated.
  • fn with ‘catchError’: Function entered as parameter to make the call to the server. The ‘catchError’ operator is added so that in the event of an error the normal flow continues, but assuming that an empty list has been returned.

If we run the application, we can see how it all works together.

Testing: configuration

Finally, we will move on to the testing phase to check the behaviour of the operator created. First, we create a series of variables that we will use in all the tests.

Marble diagrams for testing RxJs operators

We create the variables ‘LIST’ and ‘LIST_FILTERED_BY_APP’ with the server’s dummy response.
And we also create the variables ‘SOURCE_VALUES’ and ‘EXPECTED_VALUES’ where the real values to be used in the diagrams to be made are located. We will enter this mapping when creating the observables so that the actual values are output.

Next, we set up the TestScheduler:

Marble diagrams for testing RxJs operators

It is important to do this in the beforeEach to have a separate scheduler instance for each test.

Test 1: Initial value and delayed search

Marble diagrams for testing RxJs operators
Marble diagrams for testing RxJs operators

In this test we simulate the following:

  • First 1s elapses and then ‘a’ is output.
  • The values ‘ap’ and ‘app’ are then output every 50ms, which simulates user typing.

The expected result is:

  • Instantly an undefined value is expected to be output.
  • 200ms seconds later you get the response from the server and another 800ms elapses.
  • After 500ms after the input data stream has stopped changing, an undefined value will be output again.
  • And in 600ms you will have the response from the server.

* NOTE: The use of, for example, 199ms instead of 200ms is because issuing ‘u’ takes 1ms so the sum gives the expected 200ms.

For the test we have used:

  • The hot method for the main dataflow. This observable has to be declared in this form because the data producer is external to the subscription.
  • The cold method for observables that are connected to the server. These have to be this way because until the subscription occurs, the data producer is not instantiated. These observables are indicated in the ‘returnValues’ of the ‘searchFn’ to be returned as our operator invokes the function.

Also note the manual call made to the ‘flush’ method in order to force execution of the observables at that point and to be able to perform additional checks afterwards.

In this test you can see how the ‘startWith’ and ‘debounceTime’ operators are correctly configured.

Test 2: Cancel previous requests when new data

Marble diagrams for testing RxJs operators
Marble diagrams for testing RxJs operators

For the next test we have the following scenario:

  • First, 99ms elapse.
  • In the 100ms time frame, ‘app’ is output, an event that occurs before the first request to the server is resolved.

The expected result is:

  • First the undefined value is output and the first 99ms elapse.
  • Then the 500ms delay time elapses.
  • Finally, the undefined value is output again and after 600ms the server response is issued.

NOTE: For this and the following examples we do not simulate letter-by-letter outputs, as they are not relevant.

We highlight in this test that the time it takes for the server to answer has been increased in order to simulate the cancellation.

This second test verifies that the ‘switchMap’ operator in conjunction with the ‘concat’ operator works as expected.

Test 3: Do not make the call again if you are going to search for the same thing

Marble diagrams for testing RxJs operators
Marble diagrams for testing RxJs operators

For this third test we propose this scenario:

  • First 1s elapses until the input stream outputs ‘app’.
  • 1s and 100ms later the value ‘appl’ is issued, but before 500ms have elapsed ‘app’ is output again.

In this case, what is expected is:

  • The undefined value is output again, after 200ms the list with the data and another 800ms elapses.
  • After 500ms, the undefined value is output and, finally, after 600ms, the list of values is issued.

It should be noted that no more values are output; if the ‘distinctUntilChanged’ operator were removed, we would see the same call already made. So, in this test we validate that it works as expected.

Test 4: Recovering from errors

Marble diagrams for testing RxJs operators
Marble diagrams for testing RxJs operators

For the last test that we are going to explain, we consider the following situation:

  • First, 999ms elapse.
  • The value ‘1’ is then output, the value with which we are going to simulate a server failure.

The expected behaviour is:

  • In the first second the undefined value is output and the server’s response list.
  • Then 500ms elapse and the undefined value is output again.
  • After 300ms we will get an empty data list.

In this test we check that the ‘catchError’ operator is correctly configured, as it returns an empty list in case of errors.

We highlight the ‘#’ character in the observable to simulate the error.

The code can be found in our GitLab.

Conclusions

Throughout this article we have discussed reactive programming with the RxJs library and how we are using it in EDICOM for our web applications using Angular. We have focused mainly on the aspect of how to test the necessary transformations of data flows.

A different approach to test observables in a more visual way by representing their behaviour with Marble diagrams has been unveiled.