Simplifying scientific software in Python

Indexing arrays is a common task in NumPy, but it can be awkward when types do not quite match: floats versus integers, lists versus arrays. sc.findinds() finds matches anyway, and accepts multiple conditions:

sc.findinds([2,3,6,3], 3.0)  # Returns array([1, 3])

v = np.random.rand(100)
sc.findinds(v>0.4, v<0.6)    # Indices where both conditions hold

The first line is equivalent to np.nonzero(np.isclose(arr, val))[0], and the second to ((v>0.4)*(v<0.6)).nonzero()[0]. Related functions include sc.findnearest() (nearest value, whether or not it matches exactly), sc.findfirst(), sc.findlast(), and sc.smooth().

sc.save() and sc.load() handle arbitrary Python objects, including your own classes, so you can stop an analysis and pick it up later:

sc.save('results.obj', results)   # Save any Python object
results = sc.load('results.obj')  # Load it back

If you want a specific format, there is a function for that too: sc.savejson(), sc.loadjson(), sc.savetext(), sc.loadyaml(), sc.dataframe.read_csv(). And if you want to know later what produced a file, sc.savearchive() and sc.savefig() store the date, Python environment, and Git commit alongside the data or the figure, which sc.loadarchive() and sc.loadmetadata() read back.

Scientific workflows are often embarrassingly parallel, yet parallelizing them can still be a hurdle. sc.parallelize() is a shortcut to multiprocess.Pool() that accepts arguments in whichever form is most convenient:

def f(x, y):
    return x*y

out1 = sc.parallelize(f, iterarg=[(1,2), (2,3), (3,4)])
out2 = sc.parallelize(f, iterkwargs={'x':[1,2,3], 'y':[2,3,4]})
out3 = sc.parallelize(f, iterkwargs=[{'x':1, 'y':2},
                                     {'x':2, 'y':3},
                                     {'x':3, 'y':4}])

All three return [2, 6, 12]. By default the pool size is set from the number of CPUs available, but you can fix it, or allocate dynamically based on current load with sc.loadbalancer().

Sciris includes shortcuts for the parts of Matplotlib that are more fiddly than they need to be — date axes, tick formatting, mapping values onto colors:

sc.options(font='Raleway')                       # Set a custom font
x = sc.daterange('2022-06-01', '2022-12-31', as_date=True)  # Create dates
y = sc.smooth(np.random.rand(len(x))**2)*1000    # Create smoothed random numbers
c = sc.vectocolor(y, cmap='turbo')               # Set colors proportional to y

plt.scatter(x, y, c=c)  # Vanilla Matplotlib
sc.dateformatter()      # Automatic date formatting on the x-axis
sc.commaticks()         # Write 1000 as 1,000 rather than 1e3
sc.setylim()            # Set the y-axis to start at zero
sc.boxoff()             # Remove the top and right axis spines

Below are two functionally identical scripts: one written in plain Python (left), one using Sciris (right). Both sample random numbers from a user-defined function at several noise levels, save the intermediate results to disk, load them back, plot them in 3D, and report the elapsed time. The plain Python version takes about twice as many lines in total; counting only the lines that differ, and excluding comments and whitespace, it takes 33 where Sciris takes 7.

This is the output of the two scripts in the previous tab: plain Python on the left, Sciris on the right. The plots are identical apart from the colormap, which is one of several new ones that Sciris adds.