Speech Inversion Service User Guide

Speech inversion estimates articulatory movement directly from the acoustic signal (e.g., Papcun et al., 1992), without any imaging or sensors on the speaker. This makes it possible to approximate kinematic analysis in datasets that were never collected with kinematic instrumentation.

Our Speech Inversion Systems are neural networks trained to map speech audio onto kinematic ground truth. That ground truth typically comes from articulography: X-ray microbeam or electromagnetic systems that track pellets or sensors glued to the lips and tongue (Westbury et al., 1994; Rebernik et al., 2021). Our team’s systems were trained on the Wisconsin X-ray Microbeam (XRMB) dataset, which contains American English.

This service has not been validated on languages other than English. We expect it to generalize reasonably well to languages with a phonological feature distribution similar to English’s, and there is early evidence supporting that: an adult speech inversion system trained only on English data recovered tract variables in French and Russian with average correlations of .83 and .74 against articulatory ground truth, respectively. One caveat: the XRMB training data has limited coverage of back-of-tongue (dorsal) articulation, so model performance might generalize least to languages with a different distribution of dorsal productions than English.

Rather than predicting a sensor’s raw (X, Y) position in millimeters, the model predicts normalized tract variables, constriction location and constriction degree, following the Articulatory Phonology framework (Browman & Goldstein, 1992). This normalization helps tract variables generalize across speakers better than raw sensor coordinates, which shift with anatomical differences in vocal tract size (Mitra et al., 2010).

Six tract variables describe the vocal tract: lip aperture, lip protrusion, tongue tip constriction location, tongue tip constriction degree, tongue body constriction location, and tongue body constriction degree. This service also predicts three source variables describing the glottal source: fundamental frequency, periodic energy, and aperiodic energy. Adding source variables improves the accuracy of the vocal tract gesture estimates themselves (Deshmukh et al., 2005; Siriwardena & Espy-Wilson, 2023), likely because they provide timing information that constrains gesture estimation and capture nonlinear interactions between the glottal source and the vocal tract filter.

Audio file requirements

• Format: WAV files only (.wav)
• Sample rate: any sample rate is accepted; files are automatically converted to 16 kHz. Uploading your audio at 16 kHz, though not required, will allow you to upload more audio per batch.
• Duration: at least 100 milliseconds per file, no maximum
• Content: any speech is acceptable. The service does not reject recordings based on speaker characteristics, dialect, disorder type, or recording quality.
• Channels: mono preferred; stereo files are automatically converted to mono

What to avoid

• Non-speech or low-quality audio files (music, noise-only recordings) will produce output, but the estimates will not be meaningful (e.g., “garbage in, garbage out”)
• Files shorter than 100 ms will be rejected with an error message
• Files in formats other than .wav (e.g., .mp3, .m4a) will be rejected; please convert them first

Step 1 — Prepare your audio files

• If you have just one recording, you can upload the single .wav file directly; no zipping needed.
• For multiple files, collect all .wav files for a batch into one folder on your computer, then compress that folder into a single ZIP file:
• On Windows: Right-click the folder → Send to → Compressed (zipped) folder. On Mac: Right-click the folder → Compress
• The ZIP file can contain up to several hundred files. Each file will be processed separately.

Step 2 — Complete the REDCap submission form

The form will ask for:

• Your email address: results will be sent here when processing is complete
• Your audio: upload a single .wav file, or a ZIP of .wav files
• Output format: choose CSV (default), TXT, or NPY (NumPy binary)
• Once you submit, you will see a confirmation message with your Job ID. Keep this for your records.

Step 3 — Wait for your results

• Processing time depends on the number and length of files in your batch, as well as the length of the job queue. You will receive an email when your results are ready, typically within one hour. Note that the first job of the day takes a little longer as the system wakes up.
• The email will contain a download link for a ZIP of your result files. This link is valid for 7 days. After 7 days, the results are automatically deleted from our servers in accordance with our data retention policy.

For NPY files, the channel order is nasalance, PER, APER, F0, LA, LP, TBCL, TBCD, TTCL, TTCD. This order is also documented in run_metadata.json, included in the results ZIP.

• Audio files are encrypted in transit (HTTPS) and encrypted at rest on our servers (AES-256 encryption)
• Audio files are automatically deleted after 7 days. The output files (which contain only numbers, not audio) may be retained longer
• Only you (the submitting researcher) receive the download link; results are not shared with other users
• Our server logs record only a job ID, not your audio filenames or researcher email address
• This service is hosted on authorization-based infrastructure at the University of Maryland
• Please ensure your IRB protocol covers upload of speech to a third-party analysis service.

My email is not on the Speech Inversion Service allowlist. Email Nina Benway to request access. Please note that should demand outpace the available funding to run the website, priority access may be given to clinical researchers studying communication disorders.

I received an error for one of my files. The email summary will tell you which file(s) failed and why. Common reasons:

• The file is not a valid WAV (try re-exporting from your audio software)
• The file is shorter than 100 ms
• The file contains no audio signal (silence only)

Other files in your batch are still processed even if one file fails.

My download link has expired. Download links are valid for 7 days. If your link has expired, please resubmit your files.

The results don’t look right. Tract variable estimates are most reliable for clearly recorded speech at a comfortable speaking rate. Very fast speech, overlapping speakers, or very noisy recordings may produce less reliable estimates. This is a limitation of the underlying model, not an error.

One way to sanity-check output on a few files: load the trajectories in a tool like WaveSurfer alongside the audio, and screen for basic phonological plausibility. Constrictions should be timed with the audio (e.g., a stop consonant should show closure and burst where you can see and hear one), and the articulator involved should match place of articulation (e.g., bilabials should show a lip constriction, alveolars a tongue tip constriction). This works best on a handful of perceptually clear target files, and is more informative than trying to judge a whole batch at once.

If you use this service in published research, please cite at least the following papers:

Benway, N. R., Tabatabaee, S., Wang, D., Munson, B., Preston, J. L., & Espy-Wilson, C. (2026). Perceptual ratings predict speech inversion articulatory kinematics in childhood speech sound disorders. Journal of Speech, Language, and Hearing Research, 69(2), 541–561. https://doi.org/10.1044/2025_JSLHR-25-00515

Tabatabaee, S., Boyce, S., Oren, L., Tiede, M., & Espy-Wilson, C. (2025). Enhancing Acoustic-to-Articulatory Speech Inversion by Incorporating Nasality. Proc. Interspeech 2025, 325–329. https://doi.org/10.21437/Interspeech.2025-2387

References

• Attia, A. A., Siriwardena, Y. M., & Espy-Wilson, C. (2024). Improving speech inversion through self-supervised embeddings and enhanced tract variables. 2024 32nd European Signal Processing Conference (EUSIPCO), 306–310.
• Browman, C. P., & Goldstein, L. (1992). Articulatory Phonology: An overview. Phonetica, 49(3–4), 155–180.
• Chen, S., Wang, C., Chen, Z., et al. (2022). WavLM: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6), 1505–1518.
• Deshmukh, O., Espy-Wilson, C. Y., Salomon, A., & Singh, J. (2005). Use of temporal information: Detection of periodicity, aperiodicity, and pitch in speech. IEEE Transactions on Speech and Audio Processing, 13(5), 776–786.
• Mitra, V., Nam, H., Espy-Wilson, C. Y., Saltzman, E., & Goldstein, L. (2010). Retrieving tract variables from acoustics: A comparison of different machine learning strategies. IEEE Journal of Selected Topics in Signal Processing, 4(6), 1027–1045.
• Oohashi, H., Watanabe, H., & Taga, G. (2017). Acquisition of vowel articulation in childhood investigated by acoustic-to-articulatory inversion. Infant Behavior and Development, 46, 178–193.
• Papcun, G., Hochberg, J., Thomas, T. R., Laroche, F., Zacks, J., & Levy, S. (1992). Inferring articulation and recognizing gestures from acoustics with a neural network trained on X-ray microbeam data. The Journal of the Acoustical Society of America, 92(2), 688–700.
• Rebernik, T., Jacobi, J., Jonkers, R., Noiray, A., & Wieling, M. (2021). A review of data collection practices using electromagnetic articulography. Laboratory Phonology, 12(1), Article 6.
• Serkhane, J., Schwartz, J.-L., Boë, L.-J., Davis, B. L., & Matyear, C. L. (2007). Infants’ vocalizations analyzed with an articulatory model: A preliminary report. Journal of Phonetics, 35(3), 321–340.
• Siriwardena, Y. M., Boyce, S. E., Tiede, M. K., Oren, L., Fletcher, B., Stern, M., & Espy-Wilson, C. Y. (2024). Speaker-independent speech inversion for recovery of velopharyngeal port constriction degree. The Journal of the Acoustical Society of America, 156(2), 1380–1390.
• Siriwardena, Y. M., & Espy-Wilson, C. (2023). The secret source: Incorporating source features to improve acoustic-to-articulatory speech inversion. ICASSP 2023 – 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5.
• Tabatabaee, S., Boyce, S., Oren, L., Tiede, M., & Espy-Wilson, C. (2025). Enhancing Acoustic-to-Articulatory Speech Inversion by Incorporating Nasality. arXiv:2506.09231.
• Westbury, J. R., Turner, G., & Dembowski, J. (1994). X-ray microbeam speech production database user’s handbook. University of Wisconsin.