Tag: Database

Add a New Field to a Realm JS Database using Migrations

Add a New Field to a Realm JS Database using Migrations

While building my app ThinkBack I decided to use Realm as my underlying data-store. ThinkBack helps users commit notes or ideas to long term memory through passive, daily recall. To support changes and new features in the app I need to modify the existing Realm JS object model that defines the schema.

Modify Schema and Perform Migration of Existing Database

The original Note schema and Realm are defined as follows:

const noteSchema: Realm.ObjectSchema = {
  name: "Note",
  properties: {
    id: "string",
    content: "string",
    createdOn: "date",
    modifiedOn: "date",
    ...
  },
  primaryKey: "id",
}

const realm = new Realm({ path: "default", schema: [ noteSchema
... ], schemaVersion: 0, })

I wanted to modify this schema to introduce a new field called “title”. Moving forward a note must have a title and, if one isn’t provided, a default title “No title” will be applied. Adding a new field to a database is something Realm JS can handle automatically for you but applying a default value requires a little manual intervention. In order to apply a default value for existing records in the database I leveraged Realm Migrations.

Here’s what the new schema and Realm looks like:

const noteSchema: Realm.ObjectSchema = {
  name: "Note",
  properties: {
    id: "string",
title: "string", content: "string", createdOn: "date", modifiedOn: "date", ... }, primaryKey: "id", }

const realm = new Realm({ path: "default", schema: [ noteSchema ... ], onMigration: (oldRealm: Realm, newRealm: Realm) => {
const newNotes = newRealm.objects<Note>("Note"); for (let i = 0; i < newNotes.length; i++) { newNotes[i].title = "No title"; } }, schemaVersion: 1, })

According to the Mongo DB Realm docs when you create a Configuration with a schema version greater than the realm’s current version, Realm runs a migration function that you define. The function has access to the realm’s version number and incrementally updates objects in the realm to conform to the new schema. From the examples above the original schema version was 0 (default), and in the updated schema the version is 1. When Realm detects a newer version of an existing database the onMigration function defined above will be invoked, allowing opportunity to apply a default title to each note in the Realm.

How to Paginate Query Results in React Native with Realm JS (MongoDB)

How to Paginate Query Results in React Native with Realm JS (MongoDB)

While building my app ThinkBack I decided to use Realm as my underlying data-store. ThinkBack’s home screen contains a list of notes which need pagination to avoid loading everything unnecessarily.

From the Realm docs:

Pagination & Limits

Some queries only need to access a subset of all objects that match the query. Realm’s lazy-loaded collections only fetch objects when you actually access them, so you do not need any special mechanism to limit query results.

For example, if you only want to find 10 matching objects at a time (such as in a paged product catalog) you can just access ten elements of the results collection. To advance to the next page, access the next ten elements of the results collection starting at the index immediately following the last element of the previous page.

Lazy-loading removes the need to build paging logic into the query itself and instead defers pagination or limiting results to when you interact with the result set. In other words, paging is handled at the time you want to access/materialize the objects into memory rather than the time you issue a query to Realm.

This post aims to demonstrate this concept through a simplified implementation.

End to End Paging Implementation

The following implementation will load a series of “notes” from Realm, apply pagination logic and present the results in a FlatList to the user. I tend to prefer a layer of abstraction between the UI and the underlying data store, this keeps a clean separation between the front-end and any business logic that may exist.

To support this we’ll create the following:

  • RealmStore.ts, which creates an instance of the Realm datastore.
  • Note.ts, which defines the schema of the Note table in Realm.
  • NoteService.ts, which will contain logic for querying Realm and paginating the results.
  • App.tsx, which will store paging “state” (i.e. what page we’re on, page size), interact with the NoteService class to load paged notes and present them in a FlatList.

The structure of these files in the project is as follows:

root/
├─ realm/
│ ├─ Note.ts
│ ├─ NoteService.ts
│ ├─ RealmStore.ts
├─ App.tsx

To follow along locally, initialize a new React native app and install Realm JS.

Note.ts

This class defines the schema of a Realm object and will ultimately map to a table in Realm.

import uuid from 'react-native-uuid';
import Realm from 'realm';

export interface NoteProps {
  id?: string;
  content: string;
  createdOn?: Date;
  modifiedOn?: Date;
}

export default class Note {
  public id: string;
  public content: string;
  public createdOn?: Date;
  public modifiedOn?: Date;

  constructor({
    id = uuid.v4().toString(),
    content,
    createdOn,
    modifiedOn,
  }: NoteProps) {
    const now = new Date();

    this.id = id;
    this.content = content;
    this.createdOn = createdOn ?? now;
    this.modifiedOn = modifiedOn ?? now;
  }

  static schema: Realm.ObjectSchema = {
    name: 'Note',
    properties: {
      id: 'string',
      content: 'string',
      createdOn: 'date',
      modifiedOn: 'date',
    },
    primaryKey: 'id',
  };
}

RealmStore.ts

This class creates a new instance of the Realm which we’ll use in NoteService.ts to interact with the datastore.

import Note from './Note';
import {Realm} from '@realm/react';

export default new Realm({
  path: 'default',
  schema: [Note.schema],
  deleteRealmIfMigrationNeeded: __DEV__,
});

NoteService.ts

This class is responsible for loading data from the datastore and returning a paged result based on the paging properties.

Line 20 obtains a reference to the Notes Realm object but doesn’t actually issue any queries yet thanks to lazy loading which defers queries to only when you need them (accessing an object). This is effectively saying “give me access to all Notes but don’t load them into memory yet until I figure out which ones I want”.

Line 25 calls getPagedCollection which uses the provided paging options (e.g. page size, current page) to access a subset of the Realm Notes. Calling slice on the Realm Notes array is a form of access as we’re now directly interacting with individual notes. This issues a query against the Realm store and loads the subset of notes being accessed. For example, if 100 notes existed in array then array.slice(0, 10) would cause the first 10 notes to be loaded.

import {RealmObject} from 'realm/dist/public-types/Object';
import Note from './Note';
import realm from './RealmStore';

export interface PagingOptions {
  page: number;
  pageSize: number;
  hasNext: boolean;
}

export interface PagedCollection {
  result: Array;
  pagingOptions: PagingOptions;
}

export class NoteService {
  getPagedNotes(pagingOptions: PagingOptions): PagedCollection {
    // Lazy load the notes. These aren't materialized yet so no paging needs to occur
    // at the "realm" level.
    let notes = realm.objects('Note');
    notes = notes.sorted([['createdOn', false]]);

    // Access a subset of the notes based on the paging options (e.g. page, page size). This will
    // cause actual loading/materialization of the matching object(s) from the datastore.
    const pagedResult = this.getPagedCollection(notes, pagingOptions);
    console.log(
      `Paged Notes Count: ${
        pagedResult.result.length
      }, Paging options: ${JSON.stringify(pagedResult.pagingOptions)}`,
    );

    return pagedResult;
  }

  getPagedCollection(
    array: Realm.Results<RealmObject<T, never> & T>,
    pagingOptions: PagingOptions,
  ): PagedCollection {
    const {pageSize} = pagingOptions;
    let currentPage = pagingOptions.page;
    const totalPages = Math.ceil(array.length / pageSize);

    if (currentPage < 1) {
      currentPage = 1;
    }

    if (currentPage > totalPages) {
      currentPage = totalPages;
    }

    const start = (currentPage - 1) * pageSize;
    let end = start + pageSize;
    if (end > array.length) {
      end = array.length;
    }

    return {
      result: array.slice(start, end),
      pagingOptions: {
        ...pagingOptions,
        hasNext: end !== array.length,
      },
    };
  }
}

// singleton instance for convenience
export const noteService = new NoteService();

App.tsx

This component is responsible for maintaining paging state as the user interacts with the list. As the user scrolls and approaches the end of the list the next page is loaded.

Line 51 leverages the FlatList onEndReached callback which fires as the list breaches the threshold set by onEndReachedThreshold. The handler for this callback increments the current page state by one.

Line 35 of the useEffect hook declares pagingOptions.page as a dependency which means the function will be invoked any time the “page” value is modified.

Line 27 passes the current paging options state to the noteService.getPagedNotes function which returns a paged collection of notes from Realm. The loaded notes are then merged with any existing notes state which causes the component to rerender and the latest set of notes to be shown.

import React, {useState} from 'react';
import {FlatList, SafeAreaView, StatusBar, Text} from 'react-native';
import Note from './realm/Note';
import {noteService} from './realm/NoteService';

const itemStyle = {
  padding: 20,
  margin: 5,
  borderWidth: 1,
  borderColor: 'red',
};

function App(): React.JSX.Element {
  const [pagingOptions, setPagingOptions] = useState({
    page: 1,
    pageSize: 10,
    hasNext: true,
  });
  const [notes, setNotes] = useState<Note[]>([]);

  React.useEffect(() => {
    if (!pagingOptions.hasNext) {
      // no more pages remain so don't attempt to load anything.
      return;
    }

    const pagedResult = noteService.getPagedNotes(pagingOptions);
    // merge the new notes with the existing note state.
    const newNotes = [...notes, ...pagedResult.result];
    console.log(`Total notes loaded: ${newNotes.length}`);
    setPagingOptions(pagedResult.pagingOptions);
    setNotes(newNotes);

    return () => {};
  }, [pagingOptions.page]);

  return (
    <SafeAreaView>
      <StatusBar />
      <FlatList
        contentInsetAdjustmentBehavior="automatic"
        data={notes}
        extraData={notes.length}
        renderItem={item => {
          return (
            <Text key={item.item.id} style={itemStyle}>
              {item.item.content}
            </Text>
          );
        }}
        onEndReached={() => {
          if (pagingOptions.hasNext) {
            // breached the "end reached" threshold in the list and there's more notes to load,
            // increment the current page by one.
            setPagingOptions({
              ...pagingOptions,
              page: pagingOptions.page + 1,
            });
          }
        }}
        onEndReachedThreshold={0.5}
      />
    </SafeAreaView>
  );
}

export default App;

Additional Content

For convenience sake I used the below createNotes helper function to generate some notes for testing purposes in the Realm database. You can add a log statement to see the absolute path where your Realm database lives:

console.log(`Realm path: ${realm.path}`);
The following function was added to the NoteService.
createNotes() {
  for (let i = 0; i < 30; i++) {
    const createdOn = new Date();
    const note: Note = {
      id: `${i}`,
      content: `[${
        i + 1
      }] Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Habitant morbi tristique senectus et netus et malesuada fames. Etiam non quam lacus suspendisse faucibus interdum. Faucibus et molestie ac feugiat sed. Feugiat in ante metus dictum at. Morbi blandit cursus risus at ultrices mi tempus.`,
      createdOn,
      modifiedOn: createdOn,
    };
    realm.write(() => {
      realm.create<Note>('Note', note);
    });
  }
}

I added a button above the FlatList to invoke the helper function and generate the notes:

<Button
  title="Create sample notes"
  onPress={() => noteService.createNotes()}
/>