Custom Modules: the feature that makes data scientists more flexible in their AI projects with SmartPredict.

SmartPredict is an AI Platform designed for all users regardless of their AI skill level, and its features meet citizen data scientists'' and experts' needs.

Autoflow is a suitable feature for non-experts to complete an end-to-end AI project by just submitting a dataset. For more details, please see the tutorial. There is also a blog post available.

As for experts, SmartPredict ensures that they can manage their machine learning projects from end to end with no limit. Namely, the ability to configure, and customize the generated AI workflows (flowchart) that train and deploy a machine learning model or to build it from the ground up. Moreover, the most impressive is the ability to customize modules with Python code.

So, if you are an expert, discover how compelling Custom Modules are for your next  AI project with SmartPredict.

1- What is a Custom module?

" Custom modules" is an advanced SmartPredict feature that allows users to code in Python data science operations used for their AI project to train a model and deploy it.

The outcome is a module that operates like built-in ones: configurable and drag and droppable.

For programming, an integrated code editor is provided that supports all advanced functionalities such as calling API, versioning, importing framework, libraries, and packages.

The Custom Modules in the module menu:

The integrated code editor with a sample of code:

2- Why do we need a Custom module?

- Completing AI project with no limit

While the SmartPredict platform provides comprehensive modules for running a successful AI project, as an expert, you still want to apply your knowledge and want to continue to gain experience. In addition, you may be working on datasets that require specific preprocessing and want to set up your own feature engineering process. As well as you may want to build and train your own machine-learning model. This is where Custom modules come in.

- Customize and add effortlessly processing data in the deployment flowchart

In SmartPredict you fully deploy a whole customizable deployment flowchart as a web service in just one click to put the trained model in production.

Moreover, it can be updated at will even after deployment. 

Therefore, you can add Custom modules to process and transform data or to implement other techniques such as API calls to the deployment flowchart and update it at any time.

Therefore, it’s effortless to implement the predicted value in other IT environments. Likewise, the trained model can make inferences with data from other computing resources.

Note that all these features can be applied in Autoflow as the generated flowchart is customizable.

To illustrate, let's look at the following examples.

3- Example of Custom modules

- Example 1: Customizing PCA Machine learning in Iris dataset 

Let’s create a Custom Module which processes the Principal Component Analysis to the Iris dataset. The code is as follows. 

import pandas as pd
from logging import Logger
from sklearn.decomposition import PCA 
from sklearn.preprocessing import StandardScaler 
# ----------------------------------------------------------------------------------------------------------------------

# ------------------------------------------------- CONFIGURATION ------------------------------------------------------
# Follow this convention when specifying the configuration of your module.
# The name of each input and output are up to you, but the structure of each one of them should be the same.
configuration = {
    # You can create as many inputs as you want.
    'in': {
        'input_1': {
            'label': 'Input 1',
            'type': 'dataframe',
            'order': 1,
            'description': 'This is the first input.',
        }
        
    },

    # You can also create as many outputs as you want
    'out': {
        'result': {
            'label': 'Result of the operation',
            'type': 'dataframe',
            'order': 1,
            'description': 'This is the result of the operation.',
        }
    }
}
# -------------------------------------------- CUSTOM MODULE FUNCTION --------------------------------------------------

# This is a normalized function, it must take "in_data" , "param", "logger" as parameters.
# The name of this function should also contain "custom_module" in it.
def custom_module_function(in_data: dict, param: dict, logger: Logger) -> dict:
    # Retrieve the input, and the parameters of the module as follows :
    df = in_data['input_1']
    X = df.iloc[:,0:4].values 
    y = df.iloc[:,4].values
    X_std = StandardScaler().fit_transform(X)
    
    pca = PCA(n_components=2) 
    principalComponents = pca.fit_transform(X_std) 
    principalDf = pd.DataFrame(data = principalComponents , columns = ['principal component 1', 'principal component 2'])
    dfResult = pd.concat([principalDf, df[['species']]], axis = 1)
   
    logger.debug('Computing the operation...')

    # Create the logic of the custom module, this is just an example.
    
    result =  dfResult
    
    logger.info('Hey, this is the result : {} '.format(result))

    # The output of your function must be an "out_data" mapping dictionary, each key is as specified in configuration
    out_data = {
        'result': result,
    }

    # Return this map of output data
    return out_data
# ----------------------------------------------------------------------------------------------------------------------

To learn more about the principle of coding Custom modules, see the tutorial.

You just have to save, drag and drop and link it with the Iris dataset module in the Build space.

Use the Data Visualizer module to see the result.

The original Iris dataset:

The output from the PCA Custom module:

- Example 2: Calling API to send prediction in an email

In this second case, we want the trained model prediction to be sent in an email. 

Thus, we create a "Custom module" that calls an email API and adds it to the deployment flowchart as shown in the figure below. 

A deployment flowchart:

Click on the rocket icon to fully deploy the flowchart as a web service and retrieve the API provided in the Monitor space for other uses.

Otherwise, send directly a request to the Predict space to test or make inferences and get the predicted value from the trained model in your email box.

Takeaways: 

"Custom Modules" is one of SmartPredict's built-in features that allow users to create modules with Python. 

It can be used both in the build space flowchart to train and evaluate the AI model or in the deployment flowchart which will be fully deployed as a Web Service.

Advanced operations can be coded as the code editor supports advanced programming features. 

The main common use case is creating the data processing, engineering the features, building the model in the build space flowchart, and adding data processing, calling the API in the deployment flowchart.

Conclusion

SmartPredict allows data scientists to implement an AI project in a more flexible way. The custom module is one of the features that allows them to code unlimited functionality. So go to the SmartPredict AI platform and realize your end-to-end AI project seamlessly.