4. tools Package¶
4.1. bootstrap Module¶
- ni.tools.bootstrap.bootstrap(bootstrap_repetitions, model, data, test_data=, []shuffle=True, prefix='', bootstrap_data=[])[source]¶
A helper function that performs bootstrap evaluation of models.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
bootstrap_repetitions
model
data
`test_data`=[]
`shuffle`=True
- ni.tools.bootstrap.bootstrap_samples(bootstrap_data, model, data, test_data=, []shuffle=False, prefix='', boot_dim='Bootstrap Sample')[source]¶
Performs bootstrap evaluation with bootstrap data.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
- bootstrap_data
- If new data instead of trial shuffling is to be used as bootstrap data, this data should be passed here. The ni.data.data.Data Instance should contain an additional index Bootstrap Sample
model
Model to be evaluated. It needs to provide an x(), dm() and fit(x=, dm=)/fit(data) method.data
`test_data`=[]
`shuffle`=True
- `prefix`=’‘
- String that is prefixed to the results.
- boot_dim
- The ni.data.data.Data Instance should contain an additional index Bootstrap Sample or be a list. If some other index should be used as bootstrap samples, boot_dim can be set to that.
- ni.tools.bootstrap.bootstrap_time(bootstrap_repetitions, model, data, test_data=, []prefix='')[source]¶
Performs bootstrap evaluation of models.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
bootstrap_repetitions
model
Model to be evaluated. It needs to provide an x(), dm() and fit(x=, dm=) method.data
`test_data`=[]
- `prefix`=’‘
- String that is prefixed to the results.
- ni.tools.bootstrap.bootstrap_trials(bootstrap_repetitions, model, data, test_data=, []shuffle=True, prefix='', bootstrap_data=[])[source]¶
Performs bootstrap evaluation by trial shuffling.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
bootstrap_repetitions
Number of bootstrap repetitionsmodel
Model to be evaluated. It needs to provide an x(), dm() and fit(x=, dm=)/fit(data) method.data
`test_data`=[]
`shuffle`=True
- ni.tools.bootstrap.description(prefix='', additional_information='')[source]¶
Describes the common bootstrap output variables as a dictionary. additional_information will be appended to each entry, prefix will be prepended to each key.
- ni.tools.bootstrap.plotBootstrap(res, path)[source]¶
Deprecated since version 0.1: use the plot capabilities of the ni.tools.statcollector.StatCollector.
- ni.tools.bootstrap.plotCompareBootstrap(reses, path)[source]¶
Deprecated since version 0.1: use the plot capabilities of the ni.tools.statcollector.StatCollector.
4.2. progressbar Module¶
4.3. project Module¶
NI Project Management
All steps in a configuration / simulation process will be logged to some folder structure
after the simulation and even after changing the original code, the results should still be viewable / interpretable with a project viewer
batches of runs should be easy to batch interpret (characteristic plots etc.)
- metadata should contain among others:
date software versions configuration options manual comments
saving of plots/data should be done by the project manager
- class ni.tools.project.Figure(path, display=False, close=True)[source]¶
Figure Context Manager
Can be used with the with statement:
import ni x = np.arange(0,10,0.1) with ni.figure("some_test.png"): plot(cos(x)) # plots to a first plot with ni.figure("some_other_test.png"): plot(-1*np.array(x)) # plots to a second plot plot(sin(x)) # plots to the first plot again
Or if they are to be used in an interactive console:
import ni x = np.arange(0,10,0.1) with ni.figure("some_test.png",close=False): plot(cos(x)) # plots to a first plot with ni.figure("some_other_test.png",close=False): plot(-1*np.array(x)) # plots to a second plot plot(sin(x)) # plots to the first plot againBoth figures will be displayed, but the second one will remain available after the code. (But keep in mind that in the iPython pylab console, after every input, all figures will be closed)
- class ni.tools.project.Job(project, session, path, job_name='', job_number='', file='', status='initializing...', dependencies=[])[source]¶
- class ni.tools.project.Project(folder='unsaved_project', name='')[source]¶
Project Class
loads a Project folder (containing eg. a main.py file)
- class ni.tools.project.TemporaryJob(project, session, job_name)[source]¶
Bases: ni.tools.project.Job
- class ni.tools.project.TemporarySession(project)[source]¶
Bases: ni.tools.project.Session
- ni.tools.project.atoi(text)[source]¶
converts text containing numbers into ints / used by natural_keys()
- ni.tools.project.figure(path, display=False, close=True)[source]¶
Can be used with the with statement:
import ni x = np.arange(0,10,0.1) with ni.figure("some_test.png"): plot(cos(x)) # plots to a first plot with ni.figure("some_other_test.png"): plot(-1*np.array(x)) # plots to a second plot plot(sin(x)) # plots to the first plot again
Or if they are to be used in an interactive console:
import ni x = np.arange(0,10,0.1) with ni.figure("some_test.png",display=True): plot(cos(x)) # plots to a first plot with ni.figure("some_other_test.png",close=False): plot(-1*np.array(x)) # plots to a second plot plot(sin(x)) # plots to the first plot again
Both of these figures will be displayed, but the second one will remain open and can be activated again.
- ni.tools.project.natural_keys(text)[source]¶
alist.sort(key=natural_keys) sorts in human order (See Toothy’s implementation in the comments of http://nedbatchelder.com/blog/200712/human_sorting.html )
4.4. html_view Module¶
This module can generate HTML output from text or objects that provide a .html_view() function:
import ni
view = ni.View() # this is a shortcut for ni.tools.html_view.View
view.add("#1/title","This is a test")
view.add("#2/Some Example Models/tabs/",ni.model.ip.Model({'name': 'Basic Model'}))
view.add("#2/Some Example Models/tabs/",ni.model.ip.Model({'autohistory_2d':True, 'name': 'Model with Higher Dimensional Autohistory'}))
view.add("#2/Some Example Models/tabs/",ni.model.ip.Model({'rate':False, 'name': 'Model without Rate Component'}))
view.add("#3/Some Example Data/tabs/1",ni.data.monkey.Data())
view.render("this_is_a_test.html")
- class ni.tools.html_view.Figure(view, path, close=True, figsize=False)[source]¶
Figure Context Manager
Can be used with the with statement:
import ni v = ni.View() x = np.arange(0,10,0.1) with ni.tools.html_view.Figure(v,"some test"): plot(cos(x)) # plots to a first plot with ni.tools.html_view.Figure(v,"some other test"): plot(-1*np.array(x)) # plots to a second plot plot(sin(x)) # plots to the first plot again v.render("context_manager_test.html")
- class ni.tools.html_view.View(path='')[source]¶
-
- figure(path='', close=True, figsize=False)[source]¶
Provides a Context Manager for figure management
Should be used if plots are to be used in
Example:
import ni v = ni.View() x = np.arange(0,10,0.1) with v.figure("some test"): plot(cos(x)) # plot to a first plot with v.figure("some other test"): plot(-1*np.array(x)) # plot to a second plot plot(sin(x)) # plot to the first plot again v.render("context_manager_test.html")
- ni.tools.html_view.natural_keys(text)[source]¶
alist.sort(key=natural_keys) sorts in human order http://nedbatchelder.com/blog/200712/human_sorting.html (See Toothy’s implementation in the comments)
4.5. strap Module¶
- ni.tools.strap.bootstrap(bootstrap_repetitions, model, data, test_data=, []shuffle=True, prefix='', bootstrap_data=[])[source]¶
A helper function that performs bootstrap evaluation of models.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
bootstrap_repetitions
model
data
`test_data`=[]
`shuffle`=True
- ni.tools.strap.bootstrap_samples(bootstrap_data, model, data, test_data=, []shuffle=False, prefix='', boot_dim='Bootstrap Sample')[source]¶
Performs bootstrap evaluation with bootstrap data.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
- bootstrap_data
- If new data instead of trial shuffling is to be used as bootstrap data, this data should be passed here. The ni.data.data.Data Instance should contain an additional index Bootstrap Sample
model
Model to be evaluated. It needs to provide an x(), dm() and fit(x=, dm=)/fit(data) method.data
`test_data`=[]
`shuffle`=True
- `prefix`=’‘
- String that is prefixed to the results.
- boot_dim
- The ni.data.data.Data Instance should contain an additional index Bootstrap Sample or be a list. If some other index should be used as bootstrap samples, boot_dim can be set to that.
- ni.tools.strap.bootstrap_time(bootstrap_repetitions, model, data, test_data=, []prefix='')[source]¶
Performs bootstrap evaluation of models.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
bootstrap_repetitions
model
Model to be evaluated. It needs to provide an x(), dm() and fit(x=, dm=) method.data
`test_data`=[]
- `prefix`=’‘
- String that is prefixed to the results.
- ni.tools.strap.bootstrap_trials(bootstrap_repetitions, model, data, test_data=, []shuffle=True, prefix='', bootstrap_data=[])[source]¶
Performs bootstrap evaluation by trial shuffling.
A Model model is fitted with some data data, called “actual data” or “D” and subsequently on all of a number of bootstrap samples “D*_n” for n in range(bootstrap_repetitions). This yields an actual fit and bootstrap_repetitions times boot fit (or fit*) for each sample.
Use bootstrap_results for explanations on the dimensions of the result.
bootstrap_repetitions
Number of bootstrap repetitionsmodel
Model to be evaluated. It needs to provide an x(), dm() and fit(x=, dm=)/fit(data) method.data
`test_data`=[]
`shuffle`=True
- ni.tools.strap.description(prefix='', additional_information='')[source]¶
Describes the common bootstrap output variables as a dictionary. additional_information will be appended to each entry, prefix will be prepended to each key.
- ni.tools.strap.plotBootstrap(res, path)[source]¶
Deprecated since version 0.1: use the plot capabilities of the ni.tools.statcollector.StatCollector.
- ni.tools.strap.plotCompareBootstrap(reses, path)[source]¶
Deprecated since version 0.1: use the plot capabilities of the ni.tools.statcollector.StatCollector.
4.6. statcollector Module¶
- class ni.tools.statcollector.StatCollector(stat_init={})[source]¶
A class to collect statistics about models. It can be used to analyse nested models, as slashes in the name are interpreted as submodels.
Example:
>>> rate model/0 >>> rate model/1 >>> rate model/2 >>> rate model/3 >>> nested model/0 >>> nested model/1 >>> nested model/2 >>> nested model/3 >>> nested model/0/1 >>> nested model/0/2 >>> nested model/0/3 >>> nested model/0/2/1 >>> nested model/0/2/3 >>> nested model/0/2/3/1
This example could be generated by fitting a model with a certain number of crosshistory cells. In each iteration the best model is extended by another cell. The nested model can then be evaluated whether it has increasing likelihood and/or eic, aic or other statistics:
>>> stats.getModelsOnPath(['nested model',0,2,3,1]).get('eic') [ -1023, -1020, -900, -950 ]
- self.stats: a dict of lists, each list containing dicts with:
- name llf eic, aic, eice, complexity (optional) additional - a dict with more information (ignored for now)
- addNode(name, data={})[source]¶
adds the node name with the attributes in the dictionary dic. If name exists, it wll be overwritten.
- getTree(substitution_patterns=[])[source]¶
returns a tuple used by plotTree to plot a tree representation of the nodes.
- html_view()[source]¶
Generates an html_view of this object.
Example:
stats.html_view().render('stats_file.html')
- keys()[source]¶
returns the name of all nodes. Synonym of StatCollector.nodes()
- load(filename)[source]¶
Loads the StatCollector saved to filename.
This file should be a pickled dictionary.
- loadList(filenames)[source]¶
loads a list of files. (Alias of StatCollector.load_list())
- nodes()[source]¶
returns the name of all nodes. Synonym of StatCollector.keys()
- plotTree(dim, substitution_patterns=, []line_kwargs={}, marker_kwargs={}, right_to_left=False)[source]¶
Plots a tree of the nodes, using dim as the height, if the node contains dim.
Slashes in the model names will be used as the different levels in the tree. The order of the parts between the slashes is ignored for now (3/4 and 4/3 are the same).
substitution_patterns may contain substitution patterns (used to connect nodes) for re.sub as a three tuple (pattern, substitute, color), where color is the color that will be assigned to this connection
line_kwargs and marker_kwargs can contain arguments in a dictionary to alter the options to set lines or markers. The dictionary will be passed on to the plot function.
right_to_left determines, whether the plot is plotted from left to right (default) or the other way around (right_to_left = True).
- pull_from_inner_dict(from_dictionary='statistics', from_key='bic', to_key='BIC')[source]¶
If a dictionary is added as a dimension, this function can pull values from that dictionary and add them to each node that has the specific dictionary. As the bootstrap functions add the statistics dictionary of the model fit to the node, this function has to be used to eg. access the BIC criterion (which is why this is the default from and to keys).
- rename(pattern, substitution)[source]¶
Renames nodes with the regex pattern pattern just like re.sub().
Example to rename different number of knots to a tree, where each the increase in knots is counted as a submodel:
statsr = stats.rename(r'50','30/50').rename(r'30','20/30').rename(r'20','10/20').rename(r'10','5/10').rename(r'5','3/5').rename(r'30/3/50','30/50')
- rename_value_to_tree(value=-1)[source]¶
Renames nodes, such that increases of one value are counted as submodels
value is the index (of the slash splitted node name) of the value that is to be replaced. The default is -1, ie. the last value.
Example:
import ni stats = ni.StatCollector() stats.addNode('Model 0/10',{'a':100}) stats.addNode('Model 0/20',{'a':100}) stats.addNode('Model 0/30',{'a':100}) stats.addNode('Model 0/50',{'a':100}) stats.addNode('Model 0/1000',{'a':100}) stats.rename_value_to_tree().plotTree('a')
The example will rename the last node to ‘Model 0/10/20/30/50/1000’
- ni.tools.statcollector.atoi(text)[source]¶
converts text containing numbers into ints / used by natural_keys()
- ni.tools.statcollector.listToPath(name)[source]¶
Creates a string that joins a list together with slashes. The list can contain strings and numbers.
- ni.tools.statcollector.natural_keys(text)[source]¶
alist.sort(key=natural_keys) sorts in human order (See Toothy’s implementation in the comments of http://nedbatchelder.com/blog/200712/human_sorting.html )