Scripting allows you fine-grained control of what data will be sent to an upstream, or what data can be read or written by an access key. In addition to filtering the dataset to remove items, you can also transform the statements.
The script must return an array of valid xAPI statements, or an empty array. The LRS processes each xAPI statement through the VQL pipeline and collects the results into an array, so the array return type is implicit. However, you must ensure that the logic in the query does not modify the incoming data such that it is no longer valid xAPI.
Whenever a script generates an error, the input array is passed without modification. This means that an error in your script will never cause data loss. However, if you rely on a script to anonymize or control data access, a script error might expose data. Script errors can be reviewed in the Logs section of the LRS.
The below example filters the data. When statements are received, if they are any actor.name other than Rob C, they will pass, either to the upstream, to the database (when attached to the "write" command on an access key), or the API results (when attached to "read").
{ "filter": {
"actor.name": {"$ne": "Rob C"}},
"process": []}How to Append Data
You can also append additional data. This adds an extension to all statements. Note that we must manually escape this extension name
{ "filter": {},
"process": [{
"$addFields": {
"context.extensions.http://myExtension*`*com": "Some Value"}}]}How to Anonymize Actor Data
You can reference existing values and modify them with expressions. This script makes the actor anonymous in a deterministic way.
{ "filter": {},
"process": [{
"$addFields": {
"actor": {
"account": {
"homePage":"http://anonymous.com",
"name": {"$hash": "$actor"}}}}}]}How to Conditionally Alter Data
You can use a switch expression to conditionally apply some transformation. This script makes the actor anonymous when the actor.name property is Rob C. Otherwise, the actor is left as is.
{ "filter": {},
"process": [{
"$addFields": {
"actor": {
"$switch": {
"branches": [{
"case": {"$eq": ["$actor.name", "Rob C"]},
"then": {
"account": {
"homePage": "http://anonymous.com",
"name": {"$hash": "$actor"}}}}],
"default": "$actor"}}}}]}When You Can Use Saved Scripts
Now that you know how to create a VQL script, you need to know when you can use it in the LRS.
Statement Forwarding
As mentioned, you can use saved VQL scripts when forwarding statements to another store. Typically,
LRS users do this to keep two copies of the xAPI data: one with the original, “noisy” data, and another with data transformed in some important way (e.g., filtered, anonymized, etc.).
To use a saved VQL script when forwarding statements:
- In the left menu, click Management > Statement Forwarding.
- Click the Create Forwarding Rule button.
- In the Create a New Upstream LRS dialog, select or enter all the needed information.
- At the bottom, in the Transform Script pull-down menu, select the saved script you want to use.
- Click the Create Upstream LRS button.
Immediately, for all statements received in the first store, the LRS will forward to the second and apply the script you added.
Access Key Write/Read
To use a saved VQL script when using an access key to write to your LRS, or to read from it.
- In the left menu, click Management > Access Keys.
- To use a script in an existing key, click the More button (
) in the lower-left of the card.
- In the Edit Access Key dialog, make any other changes as needed.
- At the bottom, in the Write Transform Script or Read Transform Script pull-down menu, select the saved script you want to use.
- Click the Save button.
From this point on, when your LRS processes any xAPI statements using that key, it will also apply the script you added.
JavaScript Mode
Due to security considerations, JavaScript mode is no longer available.
Some logic can be difficult to express in VQL. You may write imperative JavaScript. JavaScript executed by our servers runs in a sandbox JavaScript interpreter. Because of this important security consideration, execution can have low performance. If performance is a concern, rewrite your logic in VQL. The JavaScript environment has no access to any global state, NPM modules, external scripts, HTTP, DOM or any other resources — it can only be used to express logic over strings, objects and numbers.
When writing JavaScript scripts, there are two special case variables to be aware of:
$input — this is the array of input statements
$output — this is the resulting array. You must populate this with an array value
We do expose a very limited set of functions to the interpreter for your convenance. You can use normal JavaScript language features like Math.random or Array.isArray
$hash — compute a hash from any JavaScript object
$uuid — generate a random UUID
The below example filters the data. When statements are received, if they are any actor.name ;other than Rob C, they will pass, either to the upstream, to the database (when attached to the "write" command on an access key), or the API results (when attached to "read"):
{
let out = [];
for(let i in $input)
{
if($input[i].actor.name !== "Rob C")
{
out.push($input[i])
}
}
$output = out;
}
You can also append additional data. This adds an extension to all statements. Note that we must manually escape this extension name, and that for simplicity's sake, this example destroys the existing extensions.
{
let out = [];
for(let i in $input)
{
$input[i].context = {
extensions: {
"http://myExtension*`*com": "Some Value"
}
}
out.push($input[i])
}
$output = out;
}
You can reference existing values and modify them with expressions. This script makes the actor anonymous in a deterministic way.
{
let out = [];
for(let i in $input)
{
$input[i].actor = {
account: {
homePage: "http://anonymous.com",
name: $hash($input[i].actor)
}
}
out.push($input[i])
}
$output = out;
}