Evaluate

../../_images/evaluateNode.png

The Clario Evaluate node is used evaluate the model performance via a model description, a model profile, and related graphs. Evaluate will work with either the Logistic Regression or Linear Regression nodes or any data with a model attribute.

Connecting Sources

When connecting a data source to the Evaluate node, the Select Connectors dialog will pop up and require one or two sources to be defined: a data source and/or a model source.

Two Input Data Streams: Model and Data Set

The “Model Source” connector takes a data stream from either a Linear or Logistic node. This data set contains the model information only.

The “Data Source” connector takes a data stream containing the actual modeling data such as dependent and predictor attribute values.

One Input Data Stream: Data Set with Model Attribute

Only the “Data Source” connector is used and takes a data set that already contains a model score (“Rank Attribute”), which is used to generate the model description, profile, and graphs.

Configuration

The Evaluate node has two configuration tabs: Configure and Select Attributes.

Configure Tab

Two Input Data Streams: Model and Data Set

../../_images/evaluate_configureTabTwoDataStream.png

The Configure Tab contains an Available Attributes list box, a Weight Attribute field, and a Settings area.

Available Attributes

The Available Attributes are all of the attributes from the incoming data stream with numeric type.

Weight Attribute

If the incoming data set has a weight attribute, drag and drop the weight attribute into the Weight Attribute Field. The Weight attribute must be a non-zero positive integer value.

Settings

Segments

Select the number of segments for the profile and the graphs. The valid range is 1-999 segments, and the default value is 10.

Sort Order

Specify the sorting order of the model score.

  • Ascending: smallest values of the model score receive a rank of 1
  • Descending: largest values of the model score receive a rank of 1 (default setting)

Model Score Name

Type in the attribute name you want for the Model Score Name. The Model Score values will be calculated based on your “Model Source” connector model equation (linear or logistic). The Model Score Name cannot be <NULL>.

One Input Data Stream: Data Set with Model Attribute

../../_images/evaluate_configureTabOneDateStream.png

The Configure Tab contains an Available Attributes list box, a Rank Attribute field, a Weight Attribute field, and a Settings area.

Available Attributes

The Available Attributes are all the attributes from the incoming data stream with numeric type.

Model Attribute

Drag and drop the attribute you will rank on from the Available Attributes list box into the Rank Attribute Field.

Weight Attribute

If the incoming data set has a weight attribute, drag and drop the weight attribute into the Weight Attribute Field. The Weight attribute must be a non-zero positive integer value.

Settings

Segments

Select the number of Segments you want for the model documentation, the profile and the graphs. The valid range is 1-999 segments, and the default value is 10.

Sort Order

Specify the sorting order of the Rank Attribute.

  • Ascending: smallest values of the Rank Attribute receive a rank of 1
  • Descending: largest values of the Rank Attribute receive a rank of 1 (default setting)

Select Attributes Tab

../../_images/evaluate_selectAttributes.png

Two Input Data Streams: Model and Data Set

Drag and drop the attributes of interest from the Available Attributes box to the Selected Attributes box. Attributes that are in the model will be available in the results along with any Selected Attributes. Put at least one attribute in the Selected Attributes list box.

One Input Data Stream: Data Set with Rank Attribute

Drag and drop the attributes that are of interest from the Available Attributes box to the Selected Attributes box. Put at least one attribute in the Selected Attributes list box.

Results

The results set will contain a different set of screens depending on how you’ve configured Evaluate:

Two Input Data Streams: Model and Data Set

Results will contain Model Equation, Model Profile, and Graph.

One Input Data Stream: Data Set with Model Attribute

Results will contain Model Profile and Graph.

Model Equation Tab

../../_images/evaluate_modelEquationTab.png

The Model Equation Tab is only available in the Evaluate results when using two input data streams.

Equation

This shows the model equation written in ClarioScript.

Model Detail

The model detail shows each attribute in the model, along with the coefficient and score contribution (computed using the Absolute Value of the standardized estimate). Note that the score contribution is a percentage value, and all values add to 100 percent.

Model Profile Tab

../../_images/evaluate_resultsProfileTab.png

This tab displays the profile of the Model/Rank Attribute, split into equal sized segments using the Model/Rank Attribute. Note the tie break method is the mean.

By default, the Segment, Rows, Model/Rank Attribute mean, and all Selected Attribute means are displayed. The [Select Columns…] button in the bottom right corner allows you to select additional columns to display in the Segment Details table. Additional columns include: values, min, and max for each attribute as well as any model attributes that were not selected in the Select Attributes tab.

Graph Tab

../../_images/evaluate_resultsGraphTab.png

Use the Graph Tab to graph any of the data in the Model Profile. The x-axis represents model segment. Select one or two attributes to display on the y-axis by using the blue and green drop downs. The blue y-axis values are on the left side of the graph, and the green y-axis values are on the right side of the graph. One common graph will compare the Dependent Attribute and Predicted Score by segment.

Output Stream

The data output stream outputs the evaluated segments sorted by attribute and segment:

Name Type Description
attribute S Attribute name
segment S Segment number or “TOTAL” for the total attribute values. If the run had null score values there will be an additional “NO SCORE” segment row
indicator S

One or more characters describing the attribute:

“S” - Selected attribute from the data source

If Data Set Stream Only:

“M” - The model attribute from the data source

If Data Set and Model Streams:

“D” - The dependent attribute from the model stream

“P” - A predictor attribute from the model stream

rows N Count of rows (or weighted rows)
values N Count of values (or weighted values)
min N Minimum value for segment or attribute
max N Maximum value for segment or attribute
mean N Mean value for segment or attribute