Code Compliance

This section outlines integration of your trained python model to the researcher platform UI.

Hyperparameters
Input → Output Mapping
Exporting to JSON

Hyperparameters

Although we are always looking to expand the frameworks and types of models we support on our platform, we have started with a relatively narrow scope. We currently support fully-connected feedforward neural networks. Additionally, the input dimensionality cannot be changed. This means that researchers must use the features that we have constructed. In the architecture portion of

🧠

Neural Networks

we explain why we made this decision.

‣

Hidden layer activation functions that we support

Sigmoid
Tanh
ReLu
Leaky ReLu
Elu

‣

Output layer activation functions that we support

Linear
Softmax

The current set of activation functions we support should be sufficient for the majority of models. However, if you would like support for others, reach out to us in Discord.

Input → Output Mapping

If you want to code up a custom agent for your model to train against, you must create a class that has a method named select_action. This method must take the state as an input, and return the index of the action the agent will take.

We find it easier to first create a mapping from action names to indices. This way we can use the action names in the select_action method (since it is easy to mix up actions when dealing with integers), and then feed the name into the mapping at the end.

actions_list = [
    "Run Left",
    "Run Right",
    "Single Punch",
    "Double Punch",
    "Defend",
    "Jump",
    "Jump Left",
    "Jump Right",
    "Jump Punch",
    "Low Kick"
]

action_to_idx_mapping = {}
for i in range(len(actions_list)):
    action_to_idx_mapping[actions_list[i]] = i

Now we can initialize the class with a simple policy as follows:

class CustomAgent():
  def __init__(self):
      self.type = "rules-based-agent"

  def select_action(self, state):
    (
        relative_distance,
        you_facing_opponent,
        opponent_facing_you,
        your_health,
        opponent_health,
        relative_strength,
        relative_speed,
        relative_defence,
        relative_accuracy
    ) = state[0]
		
		action = "Single Punch"
		if abs(relative_distance < 0.1):
				if (relative_distance < 0):
						action = "Run Right"
				else:
						action = "Run Left"

		return action_to_idx_mapping[action]

Exporting to JSON

The model export should be a JSON file and follow the following schema:

{
	"activation_function": String,
	"output_activation": String,
	"n_features": Number,
	"n_actions": Number,
	"neurons": [Number],
	"weights": [[[Number]]],
	"biases": [[[Number]]],
}

We currently support a single activation function that is applied to all hidden layers.

In the JSON file it is represented by the name of the activation function in lowercase characters. The output activation function is also represented by its name.

n_features and n_actionsrepresent the dimensionality of the inputs (state space) and outputs (action space), respectively.

neurons represents the number of neurons in each hidden layer. For example, neurons = [48, 32, 24] means that there are 48 neurons in the first layer, 32 in the second layer, and 24 in the third.

💡

In the example above 24 is not the dimensionality of the output layer. It is the dimensionality of the last hidden layer (i.e. the layer right before the output)

Finally, weights and biases are represented as a list of the weight/bias matrices.

💡

Your weight matrices should follow the standard where rows correspond to the input dimensionality and columns correspond to the output dimensionality. If you prefer to work with columns as inputs and rows as outputs, then transpose your matrices when exporting them to JSON.

The following is an example of exporting a PyTorch model:

def save_pytorch_model(model):
    weights_data = []
    bias_data = []
    for layer in model.layers:
        current_parameters = [model.tensor_to_array(params) for params in layer.parameters()]
        weights_data.append([current_parameters[0].tolist()])
        bias_data.append([current_parameters[1].tolist()]) 

    model_data = {
        "activation_function": model.activation_function.__name__,
        "output_activation": model.output_activation.__name__,
        "n_features": model.n_features,
        "n_actions": model.n_actions,
        "neurons": list(model.neurons),
        "weights": weights_data,
        "biases": bias_data
    }
    
    json_string = json.dumps(model_data)
    with open('saved_model/model_pytorch.json', 'w') as outfile:
        outfile.write(json_string)

⬅️

↩️

Return