Method
createPipelineStep POST
Copy POST

Creates a step in a given pipeline.

Arguments:

REQUIRED KEY TYPE DESCRIPTION
Yes pipelineId str The ID of the pipeline to run.
Yes stepName str The name of the step.
No functionName str The name of the Python function.
No sourceCode str Contents of a valid Python source code file. The source code should contain the transform feature group functions. A list of allowed imports and system libraries for each language is specified in the user functions documentation section.
No stepInputMappings List[PythonFunctionArgument] List of Python function arguments.
No outputVariableMappings List[OutputVariableMapping] List of Python function outputs.
No stepDependencies List[str] List of step names this step depends on.
No packageRequirements list List of package requirement strings. For example: ['numpy==1.2.3', 'pandas>=1.4.0'].
No cpuSize str Size of the CPU for the step function.
No memory int Memory (in GB) for the step function.
No timeout int Timeout for the step in minutes, default is 300 minutes.
Note: The arguments for the API methods follow camelCase but for Python SDK underscore_case is followed.

Response:

KEY TYPE DESCRIPTION
success Boolean true if the call succeeded, false if there was an error
result Pipeline
KEY TYPE Description
pipelineName str The name of the pipeline this step is a part of.
pipelineId str The reference to the pipeline this step belongs to.
createdAt str The date and time which the pipeline was created.
notebookId str The reference to the notebook this pipeline belongs to.
cron str A cron-style string that describes when this refresh policy is to be executed in UTC
nextRunTime str The next time this pipeline will be run.
isProd bool Whether this pipeline is a production pipeline.
warning str Warning message for possible errors that might occur if the pipeline is run.
createdBy str The email of the user who created the pipeline
steps PipelineStep A list of the pipeline steps attached to the pipeline.
KEY TYPE Description
pipelineStepId str The reference to this step.
pipelineId str The reference to the pipeline this step belongs to.
stepName str The name of the step.
pipelineName str The name of the pipeline this step is a part of.
createdAt str The date and time which this step was created.
updatedAt str The date and time when this step was last updated.
pythonFunctionId str The python function_id.
stepDependencies List[str] List of steps this step depends on.
cpuSize str CPU size specified for the step function.
memory int Memory in GB specified for the step function.
timeout int Timeout for the step in minutes, default is 300 minutes.
pythonFunction PythonFunction Information about the python function for the step.
KEY TYPE Description
notebookId str The unique identifier of the notebook used to spin up the notebook upon creation.
name str The name to identify the algorithm, only uppercase letters, numbers, and underscores allowed (i.e. it must be a valid Python identifier)
createdAt str The ISO-8601 string representing when the Python function was created.
functionVariableMappings dict A description of the function variables.
outputVariableMappings dict A description of the variables returned by the function
functionName str The name of the Python function to be used.
pythonFunctionId str The unique identifier of the Python function.
functionType str The type of the Python function.
packageRequirements list The pip package dependencies required to run the code
codeSource CodeSource Information about the source code of the Python function.
KEY TYPE Description
sourceType str The type of the source, one of TEXT, PYTHON, FILE_UPLOAD, or APPLICATION_CONNECTOR
sourceCode str If the type of the source is TEXT, the raw text of the function
applicationConnectorId str The Application Connector to fetch the code from
applicationConnectorInfo str Args passed to the application connector to fetch the code
packageRequirements list The pip package dependencies required to run the code
status str The status of the code and validations
error str If the status is failed, an error message describing what went wrong
publishingMsg dict Warnings in the source code
moduleDependencies list The list of internal modules dependencies required to run the code
codeSource CodeSource Information about the source code of the step function.
KEY TYPE Description
sourceType str The type of the source, one of TEXT, PYTHON, FILE_UPLOAD, or APPLICATION_CONNECTOR
sourceCode str If the type of the source is TEXT, the raw text of the function
applicationConnectorId str The Application Connector to fetch the code from
applicationConnectorInfo str Args passed to the application connector to fetch the code
packageRequirements list The pip package dependencies required to run the code
status str The status of the code and validations
error str If the status is failed, an error message describing what went wrong
publishingMsg dict Warnings in the source code
moduleDependencies list The list of internal modules dependencies required to run the code
pipelineReferences PipelineReference A list of references from the pipeline to other objects
KEY TYPE Description
pipelineReferenceId str The id of the reference.
pipelineId str The id of the pipeline for the reference.
objectType str The object type of the reference.
datasetId str The dataset id of the reference.
modelId str The model id of the reference.
deploymentId str The deployment id of the reference.
batchPredictionDescriptionId str The batch prediction description id of the reference.
modelMonitorId str The model monitor id of the reference.
notebookId str The notebook id of the reference.
featureGroupId str The feature group id of the reference.
latestPipelineVersion PipelineVersion The latest version of the pipeline.
KEY TYPE Description
pipelineName str The name of the pipeline this step is a part of.
pipelineId str The reference to the pipeline this step belongs to.
pipelineVersion str The reference to this pipeline version.
createdAt str The date and time which this pipeline version was created.
updatedAt str The date and time which this pipeline version was updated.
completedAt str The date and time which this pipeline version was updated.
status str The status of the pipeline version.
error str The relevant error, if the status is FAILED.
stepVersions PipelineStepVersion A list of the pipeline step versions.
KEY TYPE Description
stepName str The name of the step.
pipelineStepVersion str The reference to the pipeline step version.
pipelineStepId str The reference to this step.
pipelineId str The reference to the pipeline this step belongs to.
pipelineVersion str The reference to the pipeline version.
createdAt str The date and time which this step was created.
updatedAt str The date and time when this step was last updated.
status str The status of the pipeline version.
error str The error message if the pipeline step failed.
outputErrors str The error message of a pipeline step's output.
pythonFunctionId str The reference to the python function
functionVariableMappings dict The mappings for function parameters' names.
stepDependencies List[str] List of steps this step depends on.
outputVariableMappings dict The mappings for the output variables to the step.
cpuSize str CPU size specified for the step function.
memory int Memory in GB specified for the step function.
timeout int The timeout in minutes for the pipeline step.
pipelineStepVersionReferences PipelineStepVersionReference A list to the output instances of the pipeline step version.
KEY TYPE Description
pipelineStepVersionReferenceId str The id of the reference.
pipelineStepVersion str The pipeline step version the reference is connected to.
objectType str The object type of the reference.
datasetVersion str The dataset version the reference is connected to.
modelVersion str The model version the reference is connected to.
deploymentVersion str The deployment version the reference is connected to.
batchPredictionId str The batch prediction id the reference is connected to.
modelMonitorVersion str The model monitor version the reference is connected to.
notebookVersion str The notebook version the reference is connected to.
featureGroupVersion str The feature group version the reference is connected to.
status str The status of the reference
error str The error message if the reference is in an error state.
codeSource CodeSource Information about the source code of the pipeline step version.
KEY TYPE Description
sourceType str The type of the source, one of TEXT, PYTHON, FILE_UPLOAD, or APPLICATION_CONNECTOR
sourceCode str If the type of the source is TEXT, the raw text of the function
applicationConnectorId str The Application Connector to fetch the code from
applicationConnectorInfo str Args passed to the application connector to fetch the code
packageRequirements list The pip package dependencies required to run the code
status str The status of the code and validations
error str If the status is failed, an error message describing what went wrong
publishingMsg dict Warnings in the source code
moduleDependencies list The list of internal modules dependencies required to run the code
codeSource CodeSource information on the source code
KEY TYPE Description
sourceType str The type of the source, one of TEXT, PYTHON, FILE_UPLOAD, or APPLICATION_CONNECTOR
sourceCode str If the type of the source is TEXT, the raw text of the function
applicationConnectorId str The Application Connector to fetch the code from
applicationConnectorInfo str Args passed to the application connector to fetch the code
packageRequirements list The pip package dependencies required to run the code
status str The status of the code and validations
error str If the status is failed, an error message describing what went wrong
publishingMsg dict Warnings in the source code
moduleDependencies list The list of internal modules dependencies required to run the code
pipelineVariableMappings PythonFunctionArgument A description of the function variables into the pipeline.
KEY TYPE Description
variableType PythonFunctionArgumentType The type of the python function argument
name str The name of the python function variable
pipelineVariable str The name of the pipeline variable to use as the value
isRequired bool Whether the argument is required
value Any The value of the argument
codeSource CodeSource information on the source code
KEY TYPE Description
sourceType str The type of the source, one of TEXT, PYTHON, FILE_UPLOAD, or APPLICATION_CONNECTOR
sourceCode str If the type of the source is TEXT, the raw text of the function
applicationConnectorId str The Application Connector to fetch the code from
applicationConnectorInfo str Args passed to the application connector to fetch the code
packageRequirements list The pip package dependencies required to run the code
status str The status of the code and validations
error str If the status is failed, an error message describing what went wrong
publishingMsg dict Warnings in the source code
moduleDependencies list The list of internal modules dependencies required to run the code
pipelineVariableMappings PythonFunctionArgument A description of the function variables into the pipeline.
KEY TYPE Description
variableType PythonFunctionArgumentType The type of the python function argument
name str The name of the python function variable
pipelineVariable str The name of the pipeline variable to use as the value
isRequired bool Whether the argument is required
value Any The value of the argument

Exceptions:

TYPE WHEN
DataNotFoundError

pipelineId is not found.
InvalidEnumParameterError

An invalid value is passed for cpuSize.
Language:

© Copyright