Ulysses

Ulysses Sequence Parallelism

Ulysses Sequence Parallelism enables training with million-token contexts. This technology improves model performance.