Text Chunker for LLMs
Paste a long text and split it into chunks of a size you choose, with overlap, cutting by tokens, words, characters, sentences or Markdown headings. Export the chunks as JSONL, JSON or text.
- Free, no sign-up
- Nothing is sent to a server
- Worldwide
How to use Text Chunker for LLMs online
- Paste the text. Or open a file.
- Choose the unit, size and overlap. Tokens, words, characters, sentences, headings.
- Check the table. Look at the starts and lengths.
- Copy the output. JSONL, JSON or text.
Why use this free Text Chunker for LLMs
- Five ways to cut
- Exact token chunks
- Overlap control
- JSONL, JSON and text output
- Table of chunks
- Up to one million characters
Why split text
A model can read only so much at once, and search systems that fetch passages work better with pieces of a sensible size. Splitting a long document into chunks lets you feed it in turns or store pieces for retrieval. This page cuts the text in the way you choose and shows each chunk.
Five ways to cut
Cut by tokens for an exact size under an OpenAI-style tokenizer, by words or characters for a simple size, by sentences to keep whole sentences together, or by Markdown headings to keep sections together. For sentences and headings the size is counted in words, and a unit is never cut in the middle.
Overlap
Overlap repeats the end of one chunk at the start of the next, so a thought that crosses a boundary is not lost. For sentences it repeats the last whole sentences that fit within the overlap. The overlap must be smaller than the chunk size, and the page tells you if it is not.
Output
Choose JSONL, with one object on each line holding an id, the text and the character count, or a JSON list, or plain text with separators. A table lists the first fifty chunks with their length and start so you can check the cuts before you copy.
Limits
Token cuts use one tokenizer and are exact for it only. A section that is longer than the chunk size stays whole when you cut by headings, so check the longest chunk. Texts of up to one million characters are accepted, and nothing is uploaded.
Using the tool
Change the text or any option of the Text Chunker for LLMs and the answer moves straight away. Undo and redo step back through your earlier text.
Text Chunker for LLMs: frequently asked questions
Which unit should I choose for retrieval?
How large should the overlap be?
Why is a chunk larger than my size?
Does it keep Markdown?
Does the cutting send my text anywhere?
Does the Text Chunker for LLMs keep a copy of my text?
Can I chain the Text Chunker for LLMs with another tool?
Related searches for Text Chunker for LLMs
- Text Chunker for LLMs online
- free Text Chunker for LLMs
- Text Chunker for LLMs online free
- Text Chunker for LLMs for mobile
- Text Chunker for LLMs without sign up
- Text Chunker for LLMs – Text & Writing Tools
Text Chunker for LLMs is a free online tool on MAITBOX, in the Text & Writing Tools group. It runs in your browser, so what you type is not sent to a server. It works on Android and iPhone, and on Windows, Mac and Linux, with no download and no account.