NAV Navbar
typescript

Introduction

Crank is an open system for testing and validating workflows, apps, and experiences that are defined, configured, and built in SaaS platforms, rather than in code.

This documentation will eventually be merged into crank.run, but for now, if you're looking for details on how to build a Cog. You've found the right place!

Looking for overall details about Crank? Visit crank.run

Concepts

What's in a Cog?

The gRPC interface for a Cog:

syntax = "proto3";

service CogService {
  rpc GetManifest (ManifestRequest) returns (CogManifest) {}
  rpc RunStep (RunStepRequest) returns (RunStepResponse) {}
  rpc RunSteps (stream RunStepRequest) returns (stream RunStepResponse) {}f
}

A Cog is like an assertion library for a remote system. Cogs implement an API, ensuring interoperability with other Cogs. The Cog API is built on gRPC (a modern, open source, language-agnostic RPC framework) and uses protocol buffers as both the interface definition language and the data serialization format. In gRPC parlance, a Cog is a gRPC server that implements the Cog Service interface. Crank is (among other things) a gRPC client that knows how to communicate with Cogs.

The Cog Service interface consists of just 3 methods:

More details on the request and response types for these methods can be found in the protocol buffer messages reference section. For a complete introduction on gRPC and protocol buffers, see this guide.

Authentication

The Protocol Buffer Message definition for a CogManifest

message CogManifest {

  // Example: automatoninc/eloqua
  string name = 1;

  // Example: Eloqua
  string label = 5;

  // Example: 1.0.0
  string version = 2;

  repeated StepDefinition step_definitions = 3;

  repeated FieldDefinition auth_fields = 4;

}

Crank is intended to help test and validate SaaS software configurations and other remote systems, necessitating a standard way to pass authentication details from the test runner to each underlying assertion library. Crank does this flexibly, ensuring compatibility with any authentication scheme.

In gRPC terms, authentication details are passed from client to server using gRPC metadata. Each call to a Cog's RunStep or RunSteps method includes any number of authentication fields, passed on the call's metadata.

Cogs declare the authentication details they need in their manifest on the auth_fields property in the form of FieldDefinition messages.

As an example, if a Cog were to be implemented using an underlying API client library that used HTTP Basic Authentication to connect to a remote system, the Cog's manifest would declare two auth_fields field definitions: one with a username key, and another with a password key, that the Cog could parse and apply to the underlying API client on each RunStep(s) method call.

What's in a Step (Definition)

The Protocol Buffer Message definition for a StepDefinition

message StepDefinition {

  // Example: FillOutEloquaForm
  string step_id = 1;

  // Example: Fill Out an Eloqua Form
  string name = 2;

  // See protocol buffer message reference for enum values.
  Type type = 5;

  // Example: fill out Eloqua form (?<eloquaFormName>.+)
  string expression = 3;

  // See below for an abridged FieldDefinition definition
  repeated FieldDefinition expected_fields = 4;

}

One of the most important properties on a Cog's manifest is a list of StepDefinition messages. Each step definition represents an assertion or action that the Cog can make against the system that the Cog interfaces with.

A step definition consists of three string properties, used to identify the step in various contexts, as well as a type, and a list of fields that the Cog expects when the step is executed.

For a complete reference on the StepDefinition protocol buffer message, see the corresponding protocol buffer message reference section.

An abridged Protocol Buffer Message definition of a FieldDefinition

message FieldDefinition {

  // Example: eloquaFormName
  string key = 1;

  // See protocol buffer message reference for enum values.
  Optionality optionality = 2;

  // See protocol buffer message reference for enum values.
  Type type = 3;

  // Example: HTML form name of the Eloqua form
  string description = 4;

}

Field Definitions

Accompanying each step definition is an optional list of field definitions. These represent the expected data and their schema when passed in on the Cog's RunStep or RunSteps methods. In the example in this section, you would expect two fields: an eloquaFormName and eloquaFormData, an arbitrary map of field/value pairs containing HTML form submission data.

For a complete reference on the FieldDefinition protocol buffer message, see the corresponding protocol buffer message reference section.

Docker

An example Dockerfile for a node.js Cog

FROM mhart/alpine-node:12
WORKDIR /app
COPY . .
RUN npm install && npm prune --production
ENV HOST 0.0.0.0
ENV PORT 28866
EXPOSE 28866
ENTRYPOINT ["npm", "start"]

Although it's possible to write a Cog in any language that supports gRPC, it's important to be able to easily install and operate Cogs without having to download any dependencies or prerequisites. Working Cogs are packaged up as docker images, which Crank can download, install, and run.

Cogs as docker images need only conform to the following specification:

Note, you can test SSL support in your Cogs by passing the --use-ssl flag to any sub-command used to run steps. See this documentation for details on gRPC SSL/TLS support.

We recommend using the lightest-weight base images possible: base images that support your language of choice, derived from Alpine linux are ideal.

Building Cogs

The following is a guided walkthrough of how to build a Cog using tooling provided by Crank. If you're already familiar with gRPC, you may wish to bypass this walkthrough and just look at the protocol buffer reference and concepts sections... But that path is not for the faint of heart.

First thing's first: install crank if you haven't already!

Next, be sure you've got all of the tooling and frameworks you use in your language of choice* (e.g. a recent version of node.js / npm).

Scaffolding a Cog

Note: this walkthrough currently only supports typescript, but in the future, we hope to provide boilerplate for a variety of languages. If you're interested in contributing scaffolding for another language, let us know!

Scaffolding a Cog

Recommended steps for scaffolding your first Cog

# 1) Create working directory.
mkdir /path/to/hello-world-cog
cd /path/to/hello-world-cog

# 2) Run the crank cog:scaffold command and answer prompts
crank cog:scaffold --include-example-step

You will be prompted for the following details:

Installing the Cog

In order to demonstrate the use of authentication fields, the generated Cog declares a "User Agent" field in its manifest. Crank will automatically ask for authentication details at install-time, so you will be prompted for such a value during this Cog installation step. You can enter any string or skip this step.

Crank persists Cog authentication details to disk for convenience. When a Cog is uninstalled, its associated authentication details are also removed.

Testing the Cog

# 5) Run this (or similar) command to test your Cog
crank cog:step automatoninc/hello-world

Testing a Cog

Here are some combinations you can use to see how crank cog:step behaves according to the values in the API:

Passing Assertion

Failing Assertion

Assertion with Error

Note: scaffolded Cogs include scaffolded automated tests, which you can run to validate that your Cog is working as expected. The command to run tests will vary by programming language, so check the generated README.md file for details.

Replacing the API Client

Replacing the API Client

// ./src/client/client-wrapper.ts
import { YourShinyClient } from 'your-shiny-client';

export class ClientWrapper {
  // Declare a static list of required fields here. These will be exposed to
  // Crank and other clients via the Cog's GetManifest call.
  public static expectedAuthFields: Field[] = [{
    field: 'bearerToken',
    type: FieldDefinition.Type.STRING,
    description: 'Description of field.',
    help: 'Optional-but-encouraged help text, e.g. how to get/generate it.',
  }];

  protected client: YourShinyClient;

  // When RunStep or RunSteps is invoked, a new ClientWrapper instance will be
  // instantiated with GRPC Metadata. Retrieve your required auth details here.
  constructor (auth: grpc.Metadata) {
    // And authenticate your API Client by retrieving auth details here.
    const apiToken = auth.get('bearerToken').toString();
    this.client = new YourShinyClient({ token: apiToken });
  }
}

You will almost certainly want to use an API client specific to the system that you're building the Cog for, not a generic HTTP client, as is included by default in the scaffolded code. The generated code includes a generic client only in order to define an opinionated way to use API clients.

The specific steps for replacing the default HTTP client with your own client will vary according to the language used to scaffold the Cog, but generally, you will only need to make modifications in a single file, the Client Wrapper.

You will want to uninstall or otherwise remove the default API client packages, then make changes to the following portions of the Client Wrapper:

Note that if you modify the expected auth fields, you will need to run the crank registry:rebuild command in order for Crank to pick up your changes.

Using the client wrapper in steps

Adding New Public Methods

// ./src/client/client-wrapper.ts
export class ClientWrapper {
  // Any public method here is available on the client on each step class
  public async getObjectById(id: Number) {
    return this.client.object.findOne(id);
  }
}

// ./src/steps/some-step.ts
export class SomeStep {
  async executeStep(step: Step): Promise<RunStepResponse> {
    const objectId: Number = step.getData().toJavaScript().objectId;
    const response = await this.client.getObjectBydId(objectId);
  }
}

Rather than passing and using your API client directly in each step, the generated code passes the Client Wrapper to each step. We recommend you keep as much API-specific logic as possible in the wrapper, and instead expose a very simple API to your steps using public methods on the Client Wrapper itself.

This ensures that if your underlying API client changes dramatically, or you need to replace it again, you can do so with edits in just a single file.

Modifying Steps: Static Details

Modifying static step definition details

import { BaseStep, Field, StepInterface } from '../core/base-step';
import { StepDefinition } from '../proto/cog_pb';
// In scaffolded typescript code, the step class name is the Step ID.
export class ClassNameIsTheStepId extends BaseStep implements StepInterface {
  // Other static details are protected properties on the class.
  protected stepName: string = 'This is the Step Name';
  protected stepType: StepDefinition.Type = StepDefinition.Type.VALIDATION;
  protected stepExpression: string = 'capture groups (?<nameOfExpectedField>\d+) like so';
  protected expectedFields: Field[] = [{
    field: 'nameOfExpectedField',
    type: FieldDefinition.Type.NUMERIC,
    description: 'Description of this field',
    help: 'Optional-but-encouraged help text about this field.',
  }];
}

Generated code will vary by language, but making changes to steps is similar, at least in principle, regardless of language: a step is treated as best as possible like a plugin, including both static details (that define the step and are used by the Cog to generate the CogManifest), as well as the dynamic, executable code needed to run assertions (when invoked via RunStep or RunSteps).

Static details to be aware of as you add or modify steps include:

For further details, check the StepDefinition and FieldDefinition protocol buffer message reference sections.

Note: updates to static portions of a step (e.g. the step ID, name, expression, expected fields, or even net-new steps) are cached at Cog install-time and are not automatically picked up as they are added and modified.

You will need to run crank registry:rebuild in order for Crank to notice updates to static aspects of steps and step definitions.

Modifying Steps: Test Data

Retrieving test data from a step

import { Step } from '../proto/cog_pb'
// ...
async executeStep(step: Step) {
  // Use .toJavaScript() to convert the protobuf Struct to a plain object.
  const stepData: any = step.getData().toJavaScript();
  const anExpectedField: string = stepData.nameOfExpectedField;
  // ...
}

As you add/update/remove expected fields on your step, you will need to extract this new data in your step execution logic. Step data is contained on the data property of a Step, in the form of a Struct. Each language has its own way of accessing this struct; you may wish to transform it into a friendlier, native format for convenience.

The field names / keys on this struct correspond to the field keys you define on the step definition's expected field(s). The values will be serialized on the wire as the appropriate Value message according to the type you defined on the expected field definition. If you specify the type as MAP or ANYNONSCALAR, the value itself will be a Struct.

We don't recommend or necessarily support nesting of Structs below the first level.

Modifying Steps: Test Results

Setting an outcome and message on a step response

import { RunStepResponse } from '../proto/cog_pb'
import { Value } from 'google-protobuf/google/protobuf/struct_pb';
// ...
async executeStep(step: Step) {
  const response: RunStepResponse = new RunStepResponse();
  // Use the RunStepResponse.Outcome enum when setting an outcome.
  response.setOutcome(RunStepResponse.Outcome.PASSED);
  response.setMessageFormat('Step passed with argument: %s');
  // Use Value.fromJavaScript() to easily create the Value.
  response.setMessageArgs([Value.fromJavaScript('string argument')]);
  return response;
}

The logic about when a step passes, fails, or results in an error is up to you, but you must always respond with the result of your step! A response consists of basically three properties:

Note: Unlike static details of a step (e.g. its definition), the actual code that is executed when a step is run will be picked up automatically with each crank cog:step or crank cog:steps command invocation. There's no need to rebuild the registry for these types of changes.

Modifying Steps: Result Records

Adding data records to a step response

import { RunStepResponse, StepRecord } from '../proto/cog_pb'
import { Value } from 'google-protobuf/google/protobuf/struct_pb';
// ...
async executeStep(step: Step) {
  const response: RunStepResponse = new RunStepResponse();
  // ...

  // Instantiate a record, giving it an ID and a name.
  const record: StepRecord = new StepRecord();
  record.setId('UniqID');
  record.setName('Human Readable Name');

  // Add data related to the result of this step to the record.
  record.setKeyValue(Struct.fromJavaScript({
    value1: 'Some value',
    anotherOne: 123,
  }));

  // Add the record to the response.
  response.addRecords(record);

  // Note: if the name of this cog were someorg/somecog, the above
  // step response would result in Dynamic Tokens like this:
  // {{somecog.UniqID.value1}} -> "Some Value"
  // {{somecog.UniqID.anotherOne}} -> 123
  //
  // If the step outcome was not passing, a 2-column table would be
  // printed, Object Keys to the left and Values to the right.

  return response;
}

In the course of executing a step, you might have a lot of useful contextual information at your disposal that could make debugging a result easier for the end user, or be useful in subsequent steps.

You can expose this data to Crank and end-users using Step Records, which come in various flavors depending on the type of data you want to expose.

Some examples:

If your intention is to only provide extra data to help users solve problems uncovered by failed Scenarios, then adding one or more records to a Step Response (as shown in this example) is sufficient.

If, however, you wish to expose this extra metadata, regardless of the step outcome, for use as Dynamic Tokens, you will additionally want to define the list of Expected Record Definitions on the StepDefinition.

Adding a Cog Step

Adding a new step

// To add a step in scaffolded typescript code, simply add a new Step class
// in the src/steps folder and the Cog will recognize it automatically.
// It may be easiest to duplicate an existing step as a starting point.

Depending on the nature of the language used to scaffold the Cog, the process for adding a step will vary. Specifics instructions are provided here on a language-by-language basis.

Don't forget to run crank registry:rebuild so that Crank recognizes the new step.

Uninstalling the Cog

You can always uninstall your cog by running a command like the following: crank cog:uninstall name-of/your-cog. This will not impact the code you've written so far, but will make Crank forget that your Cog exists, and remove any/all authentication details associated with your Cog.

You can always manually install a Cog that you are developing locally by running crank cog:install --source=local and answering the interactive prompts.

Cog Style Guide

In order to keep the experience of building and using Cogs as consistent and intuitive as possible, we strongly recommend you adhere to the following style guide for user-facing strings in your Cogs.

In general, you are a human, people who use your Cogs and build scenarios on top of them are human, and the robots only come in after the Cog and the scenarios are all built and written. As such, avoid using robot words like assert or validate or print whenever possible.

Instead, try to find friendlier words (that still make sense in the context of your Cog) like check or show.

In addition to this general advice, here are some specific recommendations for the various user-facing strings you will expose in your Cogs.

Step Expressions

Step expressions are the most important part of your Cog's step definitions. They effectively form an API that users rely on (as a user, you wouldn't want your scenario file to suddenly stop working), so it's important to get your step expression right in the beginning.

In addition to the technical requirements associated with step expressions, you should also keep in mind the following:

Action Steps

# Expression shouldn't include BDD keywords
Bad: `When I navigate to (?<webPageUrl>.+)`
Better: `navigate to (?<webPageUrl>.+)`

# Action expression should start with verb
Bad: `marketo email (?<emailId>\d+) is sent to (?<emailAddr>.+)`
Better: `send marketo email (?<emailId>\d+) to (?<emailAddr>.+)`

Action Steps

Steps that are defined as actions should almost always start with a verb that corresponds to the action being taken, e.g. create, trigger, add, set, etc.

Validation Steps

# Use the "should" keyword (and avoid robot words)
Bad: `assert (?<count>\d+) emails? in the inbox`
Better: `there should be (?<count>\d+) emails? in the inbox`

# Global uniqueness, and exclude BDD keywords,
Bad: `then the (?<field>.+) lead field should be (?<value>.+)`
Better: `the (?<field>.+) lead field should be (?<value>.+) in marketo`

Validation Steps

Steps that are defined as validations should almost always take the form [subject of validation] should [condition of validation], or some variation of that phrase, including a should keyword.

Step Names

Step Names

# Include helpful keywords (e.g. "fill" and "form")
Bad: `Enter value into field`
Better: `Fill out a form field`

# Avoid sounding robotic
Bad: `Assert email subject text`
Better: `Check email subject line`

Although step names do not represent an API contract with your users, they are still an important way for users to find your step as they write scenarios. Keep this, and the general "users are humans" advice in mind.

Field Descriptions

Field Descriptions

# Simplify
Bad: `Value to enter into the given field`
Better: `Field Value`

# Concise, but specific enough to understand/use
Bad: `DOM Query Selector of the button to be clicked`
Better: `Button's DOM Query Selector`

# Remove detail provided by context
Bad: `SalesForce Client ID`
Better `Client ID`

Again, field descriptions (on either expected fields on step definitions or authentication fields on the Cog manifest itself) do not constitute an API, but they are important wayfinders that can improve user experience when written thoughtfully.

Field descriptions are often used as field labels in constrained interfaces (e.g. on the CLI when running crank cog:step), so maximizing meaning while minimizing length is important.

Field descriptions are always displayed in-context (including the step name for a step field, or the Cog label for an authentication field), so unlike step expressions, there's no need to include that context in your field description.

Note for authentication specifically, you can provide a link to help documentation on the Cog manifest auth_help_url property. This will be shown to users who attempt to run your Cog without proper authentication.

Protocol Buffer Reference

The following is a reference guide to all message types in the Cog protocol buffer specification, including documentation on how to use the generated code for supported languages.

CogManifest

CogManifest methods

import { CogManifest } from './proto/cog_pb';
// ...
const manifest: CogManifest = new CogManifest();

// Set the Cog's basic details.
manifest.setName('my-org/my-cog');
manifest.setLabel('My Cog')
manifest.setVersion('1.0.0');

// Add a single step definition (type StepDefinition)
manifest.addStepDefinitions(singleStepDefinition);

// Set all step definitions at once (type StepDefinition[])
manifest.setStepDefinitionsList(manyStepDefinitions);

// Add a single auth field (type FieldDefinition)
manifest.addAuthFields(singleAuthField);

// Set all auth fields at once (type FieldDefinition[])
manifest.setAuthFieldsList(manyAuthFields);

// Other helpful metadata.
manifest.setHomepage('https://github.com/my-org/my-cog');
manifest.setAuthHelpUrl('https://github.com/my-org/my-cog/blob/master/README.md#authentication');

The CogManifest represents metadata about your Cog. The details that you include in your CogManifest are used by Cog clients (like Crank) to run, interact with, and otherwise present your Cog. Key properties include:

Note: Cog Manifest details are retrieved and cached when your Cog is first installed. Any subsequent updates to Cog metadata would only be picked up after re-installation or after running crank registry:rebuild.

StepDefinition

StepDefinition Methods

import { StepDefinition } from './proto/cog_pb';
// ...
const stepDefinition: StepDefinition = new StepDefinition();

// Set the step definition's ID, name, expression, and help text (all strings)
stepDefinition.setStepId('AssertValueOfMySystemField');
stepDefinition.setName('Assert the Value of a Field');
stepDefinition.setExpression('the MySystem (?<fieldName>.+) field should have value (?<expectedValue>.+)');
stepDefinition.setHelp('This step uses the MySystem API to read field values, and make sure they are as expected.');

// Set the step's type (using an enum)
stepDefinition.setType(StepDefinition.Type.ACTION);
// The other option is StepDefinition.Type.VALIDATION

// Add a single expected field (type FieldDefinition)
stepDefinition.addExpectedFields(singleFieldDefinition);

// Add all expected fields at once (type FieldDefinition[])
stepDefinition.setExpectedFieldsList(manyFieldDefinitions);

// Add any expected records (optional).
stepDefinition.addExpectedRecords(singleExpectedRecordDef);
stepDefinition.setExpectedRecordsList(manyRecordDefinitions);

A step represents an action, assertion, or validation that can be run against a system, e.g. creating an object, asserting that a field on an object has a certain value, printing the contents of an object, or triggering an event or action in a system.

The details provided on a StepDefinition are used by Cog clients (like Crank) to run your Cog's specific steps. A step definition consists of:

StepDefinition Enums

Type

FieldDefinition

FieldDefinition methods

import { FieldDefinition } from './proto/cog_pb';
// ...
const expectedField = new FieldDefinition();

// Set the field's key (a string)
expectedField.setKey('expectedValue');

// Set the field's type (using the corresponding enum)
expectedField.setType(FieldDefinition.Type.ANYSCALAR);

// Set the optionality of the field (using the related enum)
expectedField.setOptionality(FieldDefinition.Optionality.REQUIRED);

// Provide a useful description for humans.
expectedField.setDescription('The value used when making the assertion.');

// Provide useful help text, also for humans, mostly for documentation.
expectedField.setHelp('Does not have to be provided when operator is "is set"');

A field definition represents metadata about a field that your Cog expects in order to run. Field definitions can be applied to both Steps (to define what data is required by the step itself to run) as well as the Cog itself (to define what authentication details are required for your Cog to run any steps at all).

Field definitions consist of the following properties:

FieldDefinition Enums

Optionality

Type

RunStepRequest

RunStepRequest methods

import { Step, RunStepRequest, RunStepResponse } from './proto/cog_pb';
// ...
async runStep(
  call: grpc.ServerUnaryCall<RunStepRequest>,
  callback: grpc.sendUnaryData<RunStepResponse>,
) {
  // Retrieve the step from call.request (RunStepRequest)
  const step: Step = call.request.getStep();

  // Other contextual identifiers can be pulled like this.
  const requestId: string = call.request.getRequestId();
  const scenarioId: string = call.request.getScenarioId();
  const requestorId: string = call.request.getRequestorId();
}

A RunStep Request is a simple message, passed as the sole argument to your RunStep (or as a stream to RunSteps) method(s). It consists of a Step property, as well as three string identifiers, signifiying the "scope" of a particular request, which could be used for caching, or to otherwise treat Step runs discriminately.

Step

Step methods

// Get the Step ID from the step.
const stepId: string = step.getStepId();

// Convert the step's data to a plain JS object via Struct.toJavaScript()
const stepData: any = step.getData().toJavaScript();

// Pass these details to a theoretical dispatcher.
await dispatchStep(stepId, stepData);

This represents a Step to be run by your Cog, including the data necessary to run your step. It should correspond to a Step Definition that you provided on your CogManifest.

Steps consist of two properties:

Struct and Value objects may be cumbersome to work with natively in your preferred language, so you may want to transform these into equivalent native types for convenience.

RunStepResponse

RunStepResponse methods

import { Step, RunStepResponse } from '../proto/cog_pb';
// ...
async executeStep(step: Step): Promise<RunStepResponse> {
  const response: RunStepResponse = new RunStepResponse();

  if (everythingPassed) {
    // Set the outcome using the appropriate enum value
    response.setOutcome(RunStepResponse.Outcome.PASSED);
    response.setMessageFormat('Successfully asserted %s');
    // Add a single message arg like this:
    response.addMessageArgs(Value.fromJavaScript('assertion'));
  } else {
    response.setOutcome(RunStepResponse.Outcome.FAILED);
    response.setMessageFormat('Step failed because: %s');
    // Set all message args at once, like this:
    response.setMessageArgsList([Value.fromJavaScript('reasons')]);
  }

  return response;
}

This represents the response you send to the Cog client (e.g. Crank) once your Step has finished running (on RunStep and as a stream on RunSteps methods).

You should always return a RunStepResponse, even when there is an underlying error in your code or code your Cog depends on.

RunStepResponse Enums

Outcome

StepRecord

StepRecord methods

import { RunStepResponse, StepRecord } from '../proto/cog_pb';
// ...
async executeStep(step: Step): Promise<RunStepResponse> {
  const response: RunStepResponse = new RunStepResponse();
  const record: StepRecord = new StepRecord();

  record.setId('RecordId');
  record.setName('Name of Record');

  // Use one (and only one) of the following to set data on the record.
  record.setKeyValue(/* */);
  record.setTable(/* */);
  record.setBinary(/* */)

  response.addRecords(record);
  return response;
}

This represents a piece of structured data that may be included on a Step Response. Cog clients (like crank) will render this structured data in order to help users diagnose failures or errors. This data also forms the basis for dynamic token value substitution.

The actual data that is stored and transmitted on the Step Record may take one of three forms, and are stored on the following properties:

Note: One StepRecord instance may only contain one type of data. If you need to return more, you may specify more than one record on the Run Step Response.

RecordDefinition

RecordDefinition methods

import { RecordDefinition } from './proto/cog_pb';
// ...
const expectedRecord = new RecordDefinition();

// Set the ID (corresponding to the ID of the eventual StepRecord)
expectedRecord.setId('lead');

// Set the type of data that Cog clients should expect.
expectedRecord.setType(RecordDefinition.Type.KEYVALUE);

// Set one (of potentially many) guaranteed fields.
expectedRecord.addGuaranteedFields(someExpectedFieldDefinition);

// If there may be more fields than the guaranteed fields given, set to true.
expectedRecord.setMayHaveMoreFields(true);

This represents the definition of a StepRecord's schema. Metadata provided here informs Cog clients (like crank) of what records to expect and what form they will take. This metadata is used to improve step documentation and enable dynamic token hinting in the Scenario authoring process.

RunStepResponse Enums

Type

TableRecord

TableRecord methods

import { TableRecord } from './proto/cog_pb';
// ...
const tableRecord = new TableRecord();

// Set table headers.
tableRecord.setHeaders(Struct.fromJavaScript({
  code: 'Error Code',
  name: 'Error Name',
  msg: 'Error Message',
}));

// Set table data.
tableRecord.addRows(Struct.fromJavaScript({
  code: 401,
  name: 'Incorrect Password',
  msg: 'Re-authenticate and try again.',
}))
tableRecord.addRows(Struct.fromJavaScript({
  code: 500,
  name: 'Server Error',
  msg: 'The server was unable to respond.',
}))

This represents a type of structured data record that a RunStepResponse may include. This record type is useful when you want to represent data which is multi-dimensional (e.g. has many rows/columns). In these situations, it's recommended to use this record type, rather than returning many instances of the Struct or Key/Value record type.

BinaryRecord

BinaryRecord methods

import { BinaryRecord } from './proto/cog_pb';
// ...
const binaryRecord = new BinaryRecord();

// Set the proper mimetype.
binaryRecord.setMimeType('image/png')

// Set the raw data.
binaryRecord.setData(Buffer.from('...', 'base64'));

Represents a type of structured data record that a RunStepResponse may include. This record type is useful for large, binary objects like images, flat files, or documents.

FAQ

Why gRPC and protocol buffers?

Engineers who write and maintain automated tests for their applications often have the luxury of using a test framework written in the same language as the application itself (e.g. mocha for JavaScript apps, pytest for Python, phpunit for PHP, etc).

Because Crank expands the test domain to include any application, we decided to build the system using gRPC, enabling the use of any language to implement test steps and assertions for a given system, regardless of the breadth and makeup of the developer community and ecosystem around it.

It seems like this could lead to a lot of API calls. What do?

Yes, if you build a Cog with a Step that checks an object by hitting an API endpoint, and an end-user writes a Scenario that uses the step several times in succession, without special care, you will execute just as many requests against the system's API endpoint.

Crank provides some useful metadata (Contextual IDs) that can be used to reduce overall API usage by caching some API response data in memory. Check the RunStepRequest protocol buffer documentation for details.

Is there a tool like Postman, but for interacting with gRPC services?

Yes! But the experience is less than ideal, especially when trying to encode data in the Struct and Value protocol buffer message formats that Cog leverages. Nevertheless, it's a good tool for sanity checking your Cog, and it's also open source, so you could contribute back usability improvements too.

Check out BloomRPC.

Business users don't typically know how to work on the commandline, and they definitely don't do a lot of coding. How does Crank make it easy for them to validate their workflows?

We can't make it easy for business users without first making it easy for engineers! We won't be able to address the massive quality and reliability gap in complex SaaS technology stacks without an open ecosystem and community focused on solving the problem.

typescript