[FIXED] Load Machine Learning sklearn models (RandomForestClassifier) through java and send as argument to a function in python file

Issue

I have a ML model which is trained as saved as pickle file, Randomforestclassifier.pkl. I want to load this one time using java and then execute my "prediction" part code which is written python. So my workflow is like:

1. Read Randomforestclassifier.pkl file (one time)
2. Send this model as input to function defined in "python_file.py" which is executed from java for each request
3. python_file.py has prediction code and predictions returned should be captured by java code

Please provide suggestions for this workflow requirement I have used processbuilder in java to execute python_file.py and everything works fine except for model loading as one time activity

Solution

You could use Jep.

I actually never tested the pickle module in Jep, but for your case it would be something like this:

try(Jep jep = new Jep()) {
    // Load model
    jep.eval("import pickle");
    jep.eval("with open('Randomforestclassifier.pkl', 'rb'): as f: clf = pickle.load(f)");
    Object randomForest = jep.getValue("clf");

    ...

    // Then in another context you can pass your model to your function
    jep.eval("import predictionModule");
    jep.set("arg", randomForest);
    jep.eval("result = predictionModule.use(arg)");
    Object result = jep.getValue("result");
}

Assuming you have a module named predictionModule.py which should be something like this:

import pickle

def use(model_as_bytes):
    model = pickle.loads(model_as_bytes)
    print(model)
    # do other stuff
    ...
    return prediction

Hope this helps.

Answered By - btt
Answer Checked By - Dawn Plyler (JavaFixing Volunteer)