This guide is broken up into three parts:
For NetLogo 2.0 and 2.1, the old BehaviorSpace tool from NetLogo 1.x was replaced with a new, rewritten version which, though functional, is still under development.
The new version does not yet include all the functionality of the old one. The biggest difference is that it does not include any of the data analysis capabilities the old version had. Instead, it is assumed that you will use other software (such as a spreadsheet program or scientific visualization tool) to analyze your results.
However, the new version already has the following advantages over the old one:
The documentation for the the new version is still sketchy; if you have questions, please write feedback@ccl.northwestern.edu.
BehaviorSpace is a software tool integrated with NetLogo that allows you to perform experiments with models. It runs a model many times, systematically varying the model's settings and recording the results of each model run. This process is sometimes called "parameter sweeping". It lets you explore the model's "space" of possible behaviors and determine which combinations of settings cause the behaviors of interest.
The need for this type of experiment is revealed by the following observations. Models often have many settings, each of which can take a range of values. Together they form what in mathematics is called a parameter space for the model, whose dimensions are the number of settings, and in which every point is a particular combination of values. Running a model with different settings (and sometimes even the same ones) can lead to drastically different behavior in the system being modeled. So, how are you to know which particular configuration of values, or types of configurations, will yield the kind of behavior you are interested in? This amounts to the question of where in its huge, multi-dimension parameter space does your model perform best?
For example, suppose you want speedy synchronization from the agents in the Fireflies model. The model has four sliders &mdash number, cycle-length, flash-length and number-flashes &mdash that have approximately 2000, 100, 10 and 3 possible values, respectively. That means there are 2000 * 100 * 10 * 3 = 600,000 possible combinations of slider values! Trying combinations one at a time is hardly an efficient way to learn which one will evoke the speediest synchronization.
BehaviorSpace offers you a much better way to solve this problem. If you specify a subset of values from the ranges of each slider, it will run the model with each possible combination of those values and, during each model run, record the results. In doing so, it samples the model's parameter space &mdash not exhaustively, but enough so that you will be able to see relationships form between different sliders and the behavior of the system. After all the runs are over, a dataset is generated which you can open in a different tool, such as a spreadsheet or scientific visualization application, and explore.
The idea behind BehaviorSpace is that the way to truly understand a model is to run it multiple times with different parameter (slider) settings in order to see the whole range of behaviors the system is capable of producing. Only then is it possible to investigate when and why certain behaviors arise. Isolated trials are insufficient for this purpose because you have no reason to assume that the model will always demonstrate the particular behaviors you see. It's like eating in one restaurant in New York and then claiming you've seen all that the city has to offer. By enabling you to explore the entire parameter space of a model, and thus its entire space of behaviors, BehaviorSpace can be a powerful tool for model understanding.
To begin using BehaviorSpace, open your model, then choose the BehaviorSpace item on NetLogo's Tools menu. A small window will appear containing "Edit Experiment Setup" and "Run Experiment" buttons.
Press "Edit Experiment Setup" to begin setting up your experiment.
In the dialog that appears, you'll need to specify the following information:
Vary variables as follows: This is where you specify which settings you want varied, and what values you want them to take. Settings can be sliders, switches, choosers, or any global variable in your model. You may specify values either by listing the values you want used, or by specifying that you want to try every value within a given range. For example, to give a slider named number every value from 100 to 1000 in steps of 50, you would enter:
[number [100 50 1000]]
Or, to give it only the values of 100, 200, 400, and 800, you would enter:
[number 100 200 400 800]
Be careful with the brackets here. Note that there fewer square brackets in the second example. Including or not including this extra set of brackets is how you tell BehaviorSpace whether you are listing individual values, or specifying a range.
You can vary as many settings as you want, but you must vary at least one.
Measure runs using this reporter: This is where you specify what data you want to collect from each run. You must enter a NetLogo reporter that reports the value you want. For example, if you wanted to record how the population of turtles rose and fell during each run, you would enter:
count turtles
You may only enter one reporter, but it is still possible to collect multiple measurements, by storing the measurements in a list. For example, if you wanted to record the populations of three different breeds of turtles, you could enter:
(list (count frogs) (count mice) (count birds))
Set up model with these commands: This is where you enter the commands that will be used to begin each model run. Typically, you will enter the name of a procedure that sets up the model, typically setup. But it is also possible to include other commands as well.
Step model with these commands: This is where you enter the commands that will be run to advance to the model to the next "step". Typically, this will be the name of a procedure, such as go, but you may include any commands you like.
Stop after this many steps: This lets you set a maximum length for each run. If you don't want to set any maximum, but want the length of runs to be controlled by the next setting setting instead, enter 0.
Stop if this reporter becomes true: This lets you do model runs of varying length, ending each model run when a certain condition becomes true. For example, suppose you wanted each run to last until there were no more turtles. Then you would enter:
not any? turtles
If you want the length of runs to all be of a fixed length, enter false here.
When you're done setting up your experiment, press the "OK" button, followed by the "Run Experiment" button.
A window will appear, titled "Running Experiment". In this window, you'll see a progress report of how many runs have been completed so far and how much time has passed. If the reporter you entered for "Measure runs using this reporter" reports a number, then you'll see a plot of how that number varies over the course of each run.
You can watch your model run in the main NetLogo window. (If the "Running Experiment" window is in the way, just move it to a different place on the screen.) The graphics window and plots will update as the model runs. If you don't need to see them update, then use the checkboxes in the "Running Experiment" window to turn the updating off. This will make your model run faster, so you'll get your final results sooner.
If you want to stop your experiment before it's finished, press the "Abort" button. But note that you'll lose any results that were generated up to that point.
When all the runs have finished, the experiment is complete. BehaviorSpace will then prompt you for the name of a file to save the results to. The default name is "behaviors.csv". You can change this to any name you want, but don't leave off the ".csv" part; that indicates the file is a Comma Separated Values (CSV) file. This is a plain-text data format that is readable by any text editor as well as by most popular spreadsheet and database programs.
If you open the results file in a program that displays it in tabular format, you'll see that each individual model run occupies a single column. At the top of the column, you'll see summary information about the run, such as what the settings were for that run and how many ticks the run lasted. If you used a numeric reporter to measure your runs, then you'll see the min, max, and mean values of that reporter during the run. Finally, in the "all run data" section, you'll see what the value of your reporter was at every time step during the entire run.