We have been working with Pierre Yger, author of SpyKING CIRCUS, to update our wrapper of that algorithm in order to run it in an fully automatic mode. By default, SC produced an intentional over-clustering result, expecting that the user would follow up with a merging step (either manual, automatic, or semi-automatic). Since SpikeForest can only test automated sorters we wanted to update our wrapper to include an automatic merging step that is optionally provided by SC. The present spikeforest wrapper now utilizes the spiketoolkit wrapper that calls version 0.8.2 of SpyKING CIRCUS.
Here are before/after screenshots of the spikeforest result table to show the affect of the auto merging.
Before auto-merging in SC:
After auto-merging in SC:
Our average accuracy metric certainly does not tell the entire story, but I observe that the avg. accuracy for the PAIRED_MEA64C_YGER study set has gone from 0.76 up to 0.87, and is now among the strongest sorters for that study set. The accuracy metric has gone up for other study sets as well, but for some it has gone down somewhat (e.g., SYNTH_MONOTRODE).
The website now reflects the most recent run, but the two full analyses are archived on the “archive” tab, and the before and after snapshots are also here for reference:
BEFORE (Jul 1, 2019, 6:58 AM):
AFTER (Jul 1, 2019, 2:39 PM):
It is possible to use Python notebooks to drill down on those snapshots to get a lot of details on these runs, including console outputs and all the results. (Docs and examples for doing this are in progress)