International Audio Laboratories Erlangen

On the Use of Differentiable Viterbi Decoding for Linear-Chain CRFs

This is the accompanying website for the following paper:

1. Sebastian Strahl, Johannes Zeitler, and Meinard Müller
   On the Use of Differentiable Viterbi Decoding for Linear-Chain CRFs
   In Proceedings of the European Signal Processing Conference (EUSIPCO), Accepted, 2026. Bibtex Details Code DOI

   @inproceedings{StrahlZM26_dCRF_EUSIPCO,
   author      = {Sebastian Strahl and Johannes Zeitler and Meinard M{\"u}ller},
   title       = {On the Use of Differentiable {V}iterbi Decoding for Linear-Chain {CRFs}},
   booktitle   = {Proceedings of the European Signal Processing Conference ({EUSIPCO})},
   address     = {Bruges, Belgium},
   year        = {Accepted, 2026},
   pages       = {},
   doi         = {},
   url-pdf     = {}
   url-details = {},
   url-code    = {},
   }

Abstract

Linear-chain conditional random fields~(CRFs) are widely used for sequence labeling, where local prediction scores are combined with a transition model to determine the most likely overall label sequence via Viterbi decoding. In modern learning-based systems, neural networks typically provide these local prediction scores, which are then combined by a CRF to improve temporal structure. Since Viterbi decoding is non-differentiable, it is difficult to integrate CRF-based modeling flexibly within end-to-end trainable pipelines. While prior work on dynamic programming provides a differentiable approximation of Viterbi decoding that alleviates this limitation, empirical studies of its behavior and practical use in CRF-based models remain limited. In this paper, we provide a practical description of differentiable Viterbi decoding and illustrate its behavior when applied to linear-chain CRFs. We refer to this differentiable CRF-based module as dCRF. We further demonstrate its use as an intermediate component within a larger, end-to-end trainable pipeline through a pitch class estimation case study, where dCRF serves as a module to enhance temporal structure in spectrogram-like representations. Compared to recurrent baselines, dCRF leads to similar results, while being more controllable and parameter-efficient.

Acknowledgements

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Grants No. 500643750 (MU 2686/15-1) and 521420645 (MU 2686/17-1). The authors are with the International Audio Laboratories Erlangen, a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institute for Integrated Circuits IIS.

References

1. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira
   Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
   In Proceedings of the International Conference on Machine Learning (ICML): 282–289, 2001. Bibtex

   @inproceedings{LaffertyMP01_CRF_ICML,
   author    = {John D. Lafferty and Andrew McCallum and Fernando C. N. Pereira},
   title     = {Conditional Random Fields: {P}robabilistic Models for Segmenting and Labeling Sequence Data},
   booktitle = {Proceedings of the International Conference on Machine Learning ({ICML})},
   adress    = {Williamstown, MA, USA},
   pages     = {282--289},
   year      = {2001},
   }

2. Charles Sutton and Andrew McCallum
   An Introduction to Conditional Random Fields
   Foundations and Trends in Machine Learning, 4(4): 267–373, 2012. Bibtex DOI

   @article{SuttonM12_IntroductionCRF_FTML,
   author     = {Charles Sutton and Andrew McCallum},
   title      = {An Introduction to Conditional Random Fields},
   journal    = {Foundations and Trends in Machine Learning},
   volume     = {4},
   number     = {4},
   pages      = {267--373},
   year       = {2012},
   doi        = {10.1561/2200000013},
   }

3. Andrew J. Viterbi
   Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm
   IEEE Transactions on Information Theory, 13(2): 260–269, 1967. Bibtex

   @article{Viterbi67_ViterbiAlgorithm_IETTAW,
   author  = {Andrew J. Viterbi},
   title   = {Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm},
   journal = {{IEEE} Transactions on Information Theory},
   year    = {1967},
   volume  = {13},
   number  = {2},
   pages   = {260--269}
   }

4. Arthur Mensch and Mathieu Blondel
   Differentiable Dynamic Programming for Structured Prediction and Attention
   In Proceedings of the International Conference on Machine Learning (ICML): 3459–3468, 2018. Bibtex

   @inproceedings{MenschB18_DifferentiableDynamicProgramming_ICML,
   author    = {Arthur Mensch and Mathieu Blondel},
   title     = {Differentiable Dynamic Programming for Structured Prediction and Attention},
   booktitle = {Proceedings of the International Conference on Machine Learning ({ICML})},
   address   = {Stockholmsm{\"{a}}ssan, Stockholm, Sweden},
   pages     = {3459--3468},
   year      = {2018},
   }