fix: prevent heading and def tokens from greedily capturing multiple newlines

[Puxhkar]

…nitions

Marked version:

Markdown flavor: Markdown.pl|CommonMark|GitHub Flavored Markdown|n/a

Description

• Fixes #### (if fixing a known issue; otherwise, describe issue using the following format)

Contributor

• Test(s) exist to ensure functionality and minimize regression (if no tests added, list tests covering this PR); or,
• no tests required for this PR.
• If submitting new feature, it has been documented in the appropriate places.

Committer

In most cases, this should be a different person than the contributor.

• CI is green (no forced merge required).
• Squash and Merge PR following conventional commit guidelines.

[vercel]

@servis is attempting to deploy a commit to the MarkedJS Team on Vercel.

A member of the Team first needs to authorize it.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on refining the tokenization process in marked.js by preventing heading and definition tokens from capturing excessive newlines. This adjustment ensures more accurate token representation and improves the overall parsing behavior of the library.

Highlights

• Newline Handling: This PR addresses an issue where heading and definition tokens were greedily capturing multiple newlines.
• Raw Token Values: The changes ensure that the raw property of heading and def tokens no longer includes trailing newlines.
• Test Updates: The unit tests were updated to reflect the change in the raw property of heading tokens.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
marked-website	Ready	Preview, Comment	Mar 26, 2026 4:29am

[UziTech]

I will have to test this against a couple extensions. This could be a breaking change.

[Puxhkar]

I will have to test this against a couple extensions. This could be a breaking change.

Good point, thanks for calling that out.

I’ll test this with a few extensions and edge cases (especially ones that rely on raw and newline handling) to make sure nothing breaks. If I find any issues, I can either adjust the implementation or document it properly as a breaking change.

Let me know if there are any specific extensions you’d like me to check as well.

[UziTech]

I looked at a few extensions that use the tokens and I don't see any that would break due to this change, but I still think we should consider this a breaking change since it is not really a fix.

If we want to be consistent we should also make html blocks and indented code blocks do the same thing.