Package: textrecipes 1.2.0.9000

Emil Hvitfeldt

textrecipes: Extra 'Recipes' for Text Processing

Converting text to numerical features requires specifically created procedures, which are implemented as steps according to the 'recipes' package. These steps allows for tokenization, filtering, counting (tf and tfidf) and feature hashing.

Authors:Emil Hvitfeldt [aut, cre], Michael W. Kearney [cph], Posit Software, PBC [cph, fnd]

textrecipes_1.2.0.9000.tar.gz
textrecipes_1.2.0.9000.zip(r-4.7-x86_64)textrecipes_1.2.0.9000.zip(r-4.7-arm64)textrecipes_1.2.0.9000.zip(r-4.6-x86_64)textrecipes_1.2.0.9000.zip(r-4.6-arm64)textrecipes_1.2.0.9000.zip(r-4.5-x86_64)
textrecipes_1.2.0.9000.tgz(r-4.6-x86_64)textrecipes_1.2.0.9000.tgz(r-4.6-arm64)textrecipes_1.2.0.9000.tgz(r-4.5-x86_64)textrecipes_1.2.0.9000.tgz(r-4.5-arm64)
textrecipes_1.2.0.9000.tar.gz(r-4.7-arm64)textrecipes_1.2.0.9000.tar.gz(r-4.7-x86_64)textrecipes_1.2.0.9000.tar.gz(r-4.6-arm64)textrecipes_1.2.0.9000.tar.gz(r-4.6-x86_64)
textrecipes_1.2.0.9000.tgz(r-4.6-emscripten)
manual.pdf |manual.html✨
DESCRIPTION |NEWS
card.svg |card.png
textrecipes/json (API)

# Install 'textrecipes' in R:
install.packages('textrecipes', repos = c('https://tidymodels.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/tidymodels/textrecipes/issues

Pkgdown/docs site:https://textrecipes.tidymodels.org

Datasets:

On CRAN:

Conda:

11.54 score 165 stars 1 packages 1.3k scripts 1.3k downloads 42 exports 64 dependencies

Last updated from:b8f7a569ae. Checks:15 OK. Indexed: yes.

TargetResultTimeFilesSyslog
linux-devel-arm64OK177
linux-devel-x86_64OK238
source / vignettesOK218
linux-release-arm64OK194
linux-release-x86_64OK221
macos-release-arm64OK178
macos-release-x86_64OK511
macos-oldrel-arm64OK129
macos-oldrel-x86_64OK343
windows-devel-arm64OK272
windows-devel-x86_64OK210
windows-release-arm64OK259
windows-release-x86_64OK190
windows-oldrel-x86_64OK192
wasm-releaseOK122

Exports:%>%all_tokenizedall_tokenized_predictorscount_functionsdownload_embeddings_model2vecembedding_customembedding_huggingfaceembedding_model2vecembedding_ollamangramread_embeddings_bpembread_embeddings_model2vecrequired_pkgsshow_tokensstep_clean_levelsstep_clean_namesstep_dummy_hashstep_ldastep_lemmastep_ngramstep_pos_filterstep_sentence_embeddingsstep_sequence_onehotstep_stemstep_stopwordsstep_text_normalizationstep_textfeaturestep_texthashstep_tfstep_tfidfstep_token_replacestep_tokenfilterstep_tokenizestep_tokenize_bpestep_tokenize_sentencepiecestep_tokenize_wordpiecestep_tokenmergestep_untokenizestep_word_embeddingstidytokenlisttunable

Dependencies:classcliclockcodetoolscpp11data.tablediagramdplyrfarverfuturefuture.applygenericsggplot2globalsgluegowergtablehardhatipredisobandKernSmoothlabelinglatticelavalifecyclelistenvlubridatemagrittrMASSMatrixnnetnumDerivparallellypillarpkgconfigprodlimprogressrpurrrR6RColorBrewerRcpprecipesrlangrpartS7scalesshapeSnowballCsparsevctrsSQUAREMstringistringrsurvivaltibbletidyrtidyselecttimechangetimeDatetokenizerstzdbutf8vctrsviridisLitewithr

Static embeddings with Model2Vec and BPEmb
Model2Vec | BPEmb | Training your own embeddings | Saving the recipe

Last update: 2026-11-04
Started: 2026-10-20

Sentence embeddings
The basics | Engines | Local models and hosted services | What a recipe stores | Speed, caching, and tuning | Limitations

Last update: 2026-10-20
Started: 2026-10-15

Tokenizers
The basics | Engines and tokens | The tokenizers engine | Words | Penn Treebank | Characters and character shingles | N-grams and skip n-grams | Sentences, lines, and paragraphs | Regular expressions | Word stems | Training a tokenizer | Other engines | Custom tokenizers | Choosing a tokenizer | Next steps

Last update: 2026-10-09
Started: 2026-10-09

Using textrecipes as a sparse matrix engine
Creating a sparse matrix | Steps that produce sparse data | Controlling the number of columns | Using workflows instead

Last update: 2026-10-09
Started: 2026-10-09

Cookbook - Using more complex recipes involving text
Counting select words | Removing words in addition to the stop words list | Letter distributions | TF-IDF of ngrams of stemmed tokens

Last update: 2025-04-24
Started: 2018-11-04

Under the hood - tokenlist
tokens attribute | lemma and pos attributes

Last update: 2025-04-24
Started: 2020-04-08

Working with n-grams
Only using step_tokenize() | Using step_tokenize() and step_ngram()

Last update: 2025-04-24
Started: 2020-04-08

Readme and manuals

Help Manual

Help pageTopics
Role Selectionall_tokenized all_tokenized_predictors
List of all feature counting functionscount_functions
Download a Pretrained Model2Vec Modeldownload_embeddings_model2vec
Embedding Engines for 'step_sentence_embeddings()'embedding_custom
Hugging Face Embedding Engineembedding_huggingface
Model2Vec Embedding Engineembedding_model2vec
Ollama Embedding Engineembedding_ollama
Sample sentences with emojisemoji_samples
Read Pretrained Static Embeddings From Filesread_embeddings_bpemb read_embeddings_model2vec
Show token output of recipeshow_tokens
Clean Categorical Levelsstep_clean_levels tidy.step_clean_levels
Clean Variable Namesstep_clean_names tidy.step_clean_names
Indicator Variables via Feature Hashingstep_dummy_hash tidy.step_dummy_hash
Calculate LDA Dimension Estimates of Tokensstep_lda tidy.step_lda
Lemmatization of Token Variablesstep_lemma tidy.step_lemma
Generate n-grams From Token Variablesstep_ngram tidy.step_ngram
Part of Speech Filtering of Token Variablesstep_pos_filter tidy.step_pos_filter
Sentence Embeddings of Textstep_sentence_embeddings tidy.step_sentence_embeddings
Positional One-Hot encoding of Tokensstep_sequence_onehot tidy.step_sequence_onehot
Stemming of Token Variablesstep_stem tidy.step_stem
Filtering of Stop Words for Tokens Variablesstep_stopwords tidy.step_stopwords
Normalization of Character Variablesstep_text_normalization tidy.step_text_normalization
Calculate Set of Text Featuresstep_textfeature tidy.step_textfeature
Feature Hashing of Tokensstep_texthash tidy.step_texthash
Term frequency of Tokensstep_tf tidy.step_tf
Term Frequency-Inverse Document Frequency of Tokensstep_tfidf tidy.step_tfidf
Replace Tokens in a Token Variablestep_token_replace tidy.step_token_replace
Filter Tokens Based on Term Frequencystep_tokenfilter tidy.step_tokenfilter
Tokenization of Character Variablesstep_tokenize tidy.step_tokenize
BPE Tokenization of Character Variablesstep_tokenize_bpe tidy.step_tokenize_bpe
Sentencepiece Tokenization of Character Variablesstep_tokenize_sentencepiece tidy.step_tokenize_sentencepiece
Wordpiece Tokenization of Character Variablesstep_tokenize_wordpiece tidy.step_tokenize_wordpiece
Combine Multiple Token Variables Into Onestep_tokenmerge tidy.step_tokenmerge
Untokenization of Token Variablesstep_untokenize tidy.step_untokenize
Pretrained Word Embeddings of Tokensstep_word_embeddings tidy.step_word_embeddings
Create Token Objecttokenlist