Function; Automation
In an organisation not all functions are initated by people.
In this guide we cover using Node.js to create functionality that is initiated automatically for the organisation.
The function can be initiated based a time schedule or event from another system ie when data changes.
The mydigitalstructure Javascript framework includes a Node.js module that can help with building automation apps.
It uses similar techniques to those used in the standard browser based library.
If you have any questions please email as at learn@mydigitalstructure.cloud
Let's get the tools you will need
When developing & testing we will run the Node.js engine on your local computer.
Go to nodejs.org and follow the instructions.
Go to github.com/ibcom-lab/mydigitalstructure-learn and click 'Clone or download'.
LET'S TAKE A LOOK AT THE STARTER APP
Using your text editor (as you installed in the getting started guide) open the folder with the starter app in it.
This folder includes all the files you will need to run the starter app.
learn.js is contains example code that you can use to learn from.
RUNNING THE APP ON YOUR LOCAL COMPUTER
You can use the terminal/console that comes with your computer operating system; the instructions below are based on the MacOS Terminal.
To run the learn.js example, navigate to the folder containing learn.js and type node learn.js and press
You will then see the output based on the console.log() and mydigitalstructure._util.message() calls within the learn.js code. You can now start to modify the code, save the file and then re-run node learn.js to see your changes.
To see debug information from the mydigitalstructure library, in the settings.json file set the "testing.status" and "testing.showData" to "true".
You can also add testing/debug messages in your code using mydigitalstructure._util.message(text or data, context). If the data is a JSON object it will be stringified to show in the console.
PUTTING THE APP TO WORK
Once you are happy that the code is fit to be used by the organisation, it's time to put it to work.
We are using the AWS Lambda cloud service to host the app, but you can use any service that supports the Node.js engine.
The index.js file is structured to work with the AWS lambda service. It is similar to learn.js except that all the code is wrapped in exports.handler = function (event, context) { [functional code] }. The file name can be anything, we are using index.js as this is what AWS Lambda expects by default.
To run index.js on your local computer you will need to install the lambda-local node module.
Once you have it installed, you can run your code in the terminal aslambda-local -l index.js -t 9000 -e event.json
The package.json contains all the information that the Lambda service needs to host your application - you can edit (at the least) the name, description, author & keywords.
The Node.js code needs to compressed into a zip file so it can be uploaded to Lambda.
zip -r ../mydigitalstructure-learn-lambda.zip *
zip -r ../mydigitalstructure-learn-lambda.zip * -x "node_modules/*"If you don't already have an AWS account, then sign up for the free version.
Now that you have an AWS account, you need to log on and navigate to the Lambda service page.
Click Create function and then select Author from scratch.
Enter a Function name eg mydigitalstructure-learn-app.
Make sure Node.js is selected.
Click Create function
Make sure you check the timeout setting, the default is 3 seconds!
In the Function code section, set Code entry type to Upload a .zip file.
Click the Actions button, select Upload a .zip file, and the select the zip file on your computer, then click Upload.
You will then see the code in the Lambda editor.
The Handler is set to index.handler by default and as the main file is named index.js then you can leave it as is.
You can edit the code in the editor to have a play, but just be aware that if you reload using the zip file upload the edits will be lost.
Now that the node.js code has been uploaded you can test run it.
To test the Node.js app you need to set up some test data that will be sent to the app. In this sample app the data is then logged in the mydigitalstructure console logs section as an example of a mydigitalstructure.cloud invoke method.
Click the Test button.
You can just use the Hello World template data as a start.
Enter an Event name (eg learndata) and click Create.
The default timeout for Lambda is 3 seconds. To make this a longer duration, scroll down to Basic Settings and click the Edit button and increase the timeout to say 5 minutes (maximum is 15 minutes) and click Save.
Now you can test the code by clicking the Test button again.
The code will run and you will see the console outputs and any errors. If it has run successfully, log on to the console.mydigitalstructure.cloud go to DEBUG LOGS.
Once the app tests successfully, you can create a version and associate it with an alias and schedule it to run periodically or invoke it via an URL, as required by the organisation.
It is recommended that you version your code and set up a "prod" alias, which you can use for triggers etc.
To create a version, click the Actions button in the top right hand corner and select Publish new version; enter a version description, ie "1.0.0 (01JUN2021)", and click Publish.
To create an alias, click the Actions button in the top right hand corner and select Create alias; enter a description, ie "prod", and select the version and click Save.
Then when creating triggers, click the Alias tab click the alias, ie "prod", and then add triggers etc
Repeat version creation process everytime the code is ready for production.
When the next version has been created, click on the Alias tab, click "prod" and then click Configuration, General Configuation, then Edit; select the new version and Save.
In this example we will schedule the app to run once a day at 4AM UTC.
The AWS Lambda service uses the cron & rule syntax to set up the schedule. In this example we are using cron and the expression iscron(0 4 * * ? *).
You can use this Online CRON Expression Generator to create a CRON expression, but just remember to remove the first "0" as that is seconds and AWS does not support it.
On the function details page, click the Add Trigger button and select EventBridge (CloudWatch Events).
Set Rule to Create a new rule and give it a Rule name (eg mydigitalstructure-learn-schedule-1)
Make sure Schedule expression is selected and copy and paste in cron(0 4 * * ? *) and click Add.
An example schedule expression for running every 30 minutes, starting at 5 minutes passed the hour iscron(5/30 * * * ? *).
You can set up multiple triggers for the same lambda function and provide different event data for each trigger using the Constant (JSON text) option.
Click on the trigger and click Edit, then in the Select Triggers section set up the Configure input data and then Update.
In this example we will invoke the code via an URL. The invoked function can also return data ie act as a micro-service and allowing you to create your own domain specific API.
The AWS Lambda service works with the AWS API Gateway service.
This allows you to set up a dedicated URL, which when requested by a client, invokes the code and returns data.
We recommend creating a "prod" alias that you can then select and add the trigger to.
On the function details page, click the Add Trigger button and select API Gateway.
Set API to Create an API, set API type HTTP API, set Security to Open and click Add.
Once added, click on API Gateway under Function overview and then click on the link to the API just created.
You can then see the Invoke URL, that can be used to trigger the lambda function. It has been configured as open, so any one that knows the URL can invoke it. The security is handled within the lambda functon using the mydigitalstructure.cloud security functions (see below on this below).
There is a example code for a domain specific ("proxy") API @ github.com/ibcom-lab/mydigitalstructure-learn-proxy
You can test your invocation URL using a tool like Postman.
Setting up a Custom Domain / URL
Using AWS Certificate Manager; set up a SSL certificate that matches the custom domain name, ie "api.mydomain.com".
Using API Gateway, Custom domain names; add a custom domain to the API, ie "api.mydomain.com" and select the SSL certificate created above.
Using API Gateway, Custom domain names; Click on the custom domain just created and click API Mappings, then click Configure API mappings, then select the API and Default stage and click Save.
Using API Gateway, Routes; add a "/" route and attach to the integration created above.
Then in the domain's DNS add a CNAME to the custom domain URL.
You can invoke the lambda function from within mydigitalstructure using core_other_service_aws_lambda.
Once you have set up a user using AWS IAM and created an access ID/Key, add it to mydigitalstructure, e.g.
mydigitalstructure.cloud.save({object: 'setup_aws_account', {accessid: [id], accesskey: [key]}})
and note the id returned.
Then to invoke the function, e.g.
mydigitalstructure.cloud.invoke({method: 'core_other_service_aws_lambda', {account: [id from above], lambdafunction: 'myfunction', lambdaeventdata: {hello: 'world'}}})
Any response from the function is returned as awsresponse
That's it!
mydigitalstructure methods to manage local data, data on mydigitalstructure.cloud and your apps controller code.
| Item | Description |
|---|---|
.init |
Log on to the mydigitalstructure.cloud using settings.json data. |
.add |
Add your controller code:
You can multiple by using an array.
|
.invoke |
Invoke (run) your controller code:
|
.set |
Set data into the local temporary data space:
|
.get |
Get data from the local temporary data space:
|
.cloud.logon |
Log on to the mydigitalstructure.cloud - better to use .init() and settings.json. |
.cloud.logoff |
Log off from mydigitalstructure.cloud. |
.cloud.save |
Save data on mydigitalstructure.cloud:
|
.cloud.retrieve.cloud.query |
Retrieve data from mydigitalstructure.cloud using the query language:
|
.cloud.invoke |
Invoke a method directly on mydigitalstructure.cloud:
Only use this if .retrieve() & .save() not approriate.
|
._util.message |
Send a message to the console:
|
._util.onComplete |
Invoke a callback controller when complete:
|
._util.param.get(param, field, options) |
Get a field value from an object:
Returns:
|
._util.param.set(param, field, value, options) |
Set a field value in an object:
Returns:
|