Is tokenizers safe to use?

tokenizers has no known vulnerabilities on the latest version (0.23.1). DepScope rates its health at 63/100 (moderate risk).

What is the latest version of tokenizers?

The latest version of tokenizers is 0.23.1, released 2026-04-27.

Does tokenizers have known vulnerabilities?

No known vulnerabilities on the latest version of tokenizers.

Is tokenizers deprecated?

No, tokenizers is actively maintained. Latest release: 0.23.1 (2026-04-27).

What are alternatives to tokenizers?

Popular alternatives to tokenizers in PyPI include: boto3, certifi, requests, pycparser, h11.

What license does tokenizers use?

tokenizers is licensed under Apache-2.0.

tokenizers

pypiv0.23.1

License Apache-2.0permissive100 versions12 deps

huggingface/tokenizers

/ 100

Health

use with caution

[email protected] low health (63/100) — consider alternatives

Moderate health score (63/100) — verify manually

Health breakdown0 – 100

15/25

maintenance

0/20

popularity

25/25

security

15/15

maturity

8/15

community

Vulnerabilities

none known

Maintainer trust

Active maintainers (3m)

22

Contributors (12m)

39

Primary author dominance

33%

GitHub stars

10,653

Quality signals

OSS Criticality

0.50medium

Download trend

declining(-32.6%)

Health History

Dependency Tree

License Audit

Dependencies (12)

huggingface-hub tokenizers[testing];sphinx;sphinx-rtd-theme;setuptools-rust;pytest;pytest-asyncio;requests;numpy;datasets;ruff;ty;

API access

Get this data programmatically — free, no authentication.

curl https://depscope.dev/api/check/pypi/tokenizers