We only need to create the Jupyter notebook file and Dockerfile in this section.
#Automation arraysync similar softwares install
py $ pyenv virtualenv 3.7.3 autotraining $ pyenv activate autotraining $ pip install tensorflow tensorflowjs jupyter $ pip freeze > requirements.txt To create the training scheme we will use a Jupyter notebook, and we will also need the tensorflowjs module so that we can convert the saved model into a format that TensorFlow.js can understand. Navigate to the py folder and create a new virtual environment.
#Automation arraysync similar softwares code
$ touch test.js upload.js train.js infer.js s3proxy.jsįinally, copy the boilerplate code from handler.js into each of these files, and then delete handler.js. follow prompts $ npm install using the architecture diagram as a guide, create the necessary JavaScript files that will map to the final Lambda functions. Navigate to the js folder, initialize a new Node project, and install Tensorflow.js (browser version only!). $ npm install -g serverless $ mkdir -p LambdaAutoTraining/ $ cd LambdaAutoTraining $ serverless create -template aws-nodejs Node setup A boilerplate handler ( handler.js) and configuration file ( serverless.yml) should appear. To get started, install Serverless and initialize a new project using the Node template. We will also leverage the Serverless framework, which will remain at the top-level, while the Node and Python portions will be initialized in their respective folders. Since our project will be mixed with Node Lambda files and Python EC2 files, we will separate them in the folder structure as shown below. model - Stores metadata and metrics from the training jobs.data - Will keep labeled input data for training.The actual model will use a simple (non-optimized) neural network built using TensorFlow’s Keras API.įor data storage we will create two tables in DynamoDB: Given that we will not be training within the Lambda functions, the performance hit is acceptable for prediction and therefore we will use the browser version which unpacks to 55MB.įor the underlying machine learning model we will attempt to predict a person’s comfort level based on the following input parameters:
The Node version would seem like the obvious choice, however it unpacks to 690MB(!) which makes it immediately unsuitable for Lambda. TensorFlow.js comes in a browser version and a Node version, with the latter including C++ bindings for improved performance. To mitigate this, all of the Lambda functions will be written for Node.js which will also allow us to use TensorFlow.js instead of the standard Python library. Keeping the prediction functionality in Lambda means that there still could be a size constraint due to loading TensorFlow.
0 kommentar(er)
