SpyKING CIRCUS with auto-merging (before/after)

We have been working with Pierre Yger, author of SpyKING CIRCUS, to update our wrapper of that algorithm in order to run it in an fully automatic mode. By default, SC produced an intentional over-clustering result, expecting that the user would follow up with a merging step (either manual, automatic, or semi-automatic). Since SpikeForest can only test automated sorters we wanted to update our wrapper to include an automatic merging step that is optionally provided by SC. The present spikeforest wrapper now utilizes the spiketoolkit wrapper that calls version 0.8.2 of SpyKING CIRCUS.

Here are before/after screenshots of the spikeforest result table to show the affect of the auto merging.

Before auto-merging in SC:

After auto-merging in SC:

Our average accuracy metric certainly does not tell the entire story, but I observe that the avg. accuracy for the PAIRED_MEA64C_YGER study set has gone from 0.76 up to 0.87, and is now among the strongest sorters for that study set. The accuracy metric has gone up for other study sets as well, but for some it has gone down somewhat (e.g., SYNTH_MONOTRODE).

The website now reflects the most recent run, but the two full analyses are archived on the “archive” tab, and the before and after snapshots are also here for reference:

BEFORE (Jul 1, 2019, 6:58 AM):
sha1://706e0c70467539175cfcc607070814dbdaf467ff/analysis.json

AFTER (Jul 1, 2019, 2:39 PM):
sha1://a88ce29157e2077ab2bcdc691e718c874b7c5f78/analysis.json

It is possible to use Python notebooks to drill down on those snapshots to get a lot of details on these runs, including console outputs and all the results. (Docs and examples for doing this are in progress)

1 Like

This is insupportable, SC went over TDC another time :slight_smile:

3 Likes

I fully expect TDC to make a comeback!

Cooperation is far better than competition. :slight_smile:

Indeed, let’s try to unite our efforts in such a complicated quest… I really don’t think there will be a single software for all scientific questions/dataset. So let’s try to understand pros and cons, with the nice pipeline that you are all setting up!

3 Likes

Well put! I agree that discovering the pros and cons of each software is very important and that, most likely, each approach will be useful for different cases. We are having some discussion about how we can best show the advantages and disadvantages of each software in the ‘Metric Expansion for SpikeForest’ chat room if you have more suggestions. :slight_smile:

It was a joke of course.

2 Likes