Learn Constraints tab

This page describes use of SC-IPM's Learn Constraints tab. See App Startup and Main Window for introductory notes.

Overview

SC-IPM's Learn Constraints tab allows the program to "learn" new constraints, based on model-characterization selections made by the user. The new constraints will ensure that Bad models are avoided and Good models are retained. Neutral models may or may not be retained.

Preparation

After following the notes on the Test Models tab, make sure that the Learn Constraints tab is selected:

The bottom bar will contain two items:

  • Modify Spec (button)
  • Save Modified Spec (button)

Note: The Save Modified Spec button is inoperative at the moment. See below for notes on the current workflow.

The lower-left pane will contain a two-column list of models, sorted (in decreasing order) by score.

Characterize Models

Click on the top model in the list (0046). The right-hand pane will display detailed information for the model:

The status of the model (Neutral) is shown at the bottom of the pane. Click on this, repetitively, and note that it cycles through Good and Bad and back to Neutral. Note also that the model's Status value in the lower-left pane changes to reflect the current setting.

This model has both exponential-growth and logistic-growth processes. Let's tell SC-IPM that this is a desired attribute. We'll do this by characterizing several models as Good or Bad, based on whether they include both processes.

Click on the status setting again, leaving it at Good. The next two models in the list (0098, 0090) also have both types of growth processes, so we set their status to Good, as well. However, the fourth model (0099) does not include an exponential-growth process. So, we set its status to Bad:

Continuing, we set three more models (0057, 0015, 0063) to Good and three others (0049, 0040, 0058) to Bad. Our final display reflects our choices. Note the color-coding of the model names (green for Good, red for Bad, black for Neutral):

Note: Characterizing multiple models according to the same attribute helps SC-IPM to know which models it needs to retain. In particular, if the Good and Bad models are differentiated by more than one attribute, SC-IPM may not know which attribute(s) to constrain.

Modify the Spec

Now that we've characterized some models, we can use the Modify Spec button to modify the specification, using constraints learned from the model characterizations:

     

The upper-left pane shows the output from the MISC (Method for Inducing Structural Constraints) subsystem. MISC read the definitions for the ten models that we characterized and doesn't appear to have had any problems with them. The lower-left pane lists six "Models Selected by Chosen Constraints". Not surprisingly, these are all models that we set to Good.

The right-hand pane has two sections, containing entries for Original Constraints (from our spec) and Suggested Constraints (that MISC created). Each entry lists the constraint's name, type, and item (ie, process) complement.

The Suggested Constraints entries also have Select / Reject links. Toggling a link, we can see the effect of its constraint. When Select is chosen, only the six Good models are shown; when Reject is chosen, all ten of the characterized models are listed:

       

Edit and Accept Constraint

Although MISC's proposed constraint (MISC-4) is incomplete, it seems like a good starting point. After editing the constraint a bit, we can accept it for testing, etc. Two edits are needed: add the logistic-growth process and give the constraint a meaningful name (eg, GROWTH-REQ). In summary, the Learn Constraints tab has given us a fine start; we simply need to tweak the results.

MISC-4 says that exponential-growth is a NECESSARY process, but says nothing about logistic-growth. This reflects the fact that all of the models we characterized (indeed, all of the best-performing models) include logistic-growth. So, MISC had no reason to include it in a constraint.

However, we don't want to test any models that lack the logistic-growth process. Constructing and testing sub-optimal models is a waste of time for both the computer and the modeler. It also lowers the overall quality of the reported results.

Fortunately, this isn't a real problem, so let's get started. Clicking on the MISC-4 constraint puts us in the Inspect Library tab. The right-hand pane contains the generated information:

Clicking on the Copy link makes an editable copy of the constraint. We can then click Edit and create a tidied-up version. We rename the constraint from MISC-4-COPY to GROWTH-REQ, then add the logistic-growth process:

Looking in the upper-left pane, we see that the constraint has been added:

No Suggested Constraints?

With heuristic code such as MISC, success is not always guaranteed. Worse, it isn't always clear why a search might fail. However, one possibility is an insufficient number of sample cases. There are several indications that MISC has failed to find a solution:

  • The upper-left pane says "0 models read".
  • There are no entries shown in the lower-left pane.
  • There are no Suggested Constraints in the right-hand pane.

If you see these symptoms, the first thing to try is adding more models. Click the Go Back button to restore your model characterization settings. Specify settings for several more models, and try the Modify Spec button again.

If this fails, you may have generated a set of models which MISC finds difficult to analyze. Try running Test Models again and see if the next (pseudo-random) set works better.

Save Current Library

After selecting or rejecting assorted constraints, we may reach a set that seems worth saving. To do this, we click the Save Current Library button in the Inspect Library tab.


This wiki page is maintained by Rich Morin, an independent consultant specializing in software design, development, and documentation. Please feel free to email comments, inquiries, suggestions, etc!

Topic revision: r22 - 04 Apr 2016, RichMorin
This site is powered by Foswiki Copyright © by the contributing authors. All material on this wiki is the property of the contributing authors.
Foswiki version v2.1.6, Release Foswiki-2.1.6, Plugin API version 2.4
Ideas, requests, problems regarding CFCL Wiki? Send us email