Publications

International Conference

2026

• Kazuki Yano, Jun Suzuki, Shinji Watanabe. Adapting Text LLMs to Speech via Multimodal Depth Up-Scaling. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH) [arXiv].
• Kazuki Yano, Shun Kiyono, Sosuke Kobayashi, Sho Takase, and Jun Suzuki. Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning. Proceedings of the Fourteenth International Conference on Learning Representations (ICLR) [OpenReview].
• Ryo Fujii, Makoto Morishita, Kazuki Yano, and Jun Suzuki. TimeMachine-bench: A Benchmark on Evaluating Model's Capability on Repository-level Migration Tasks. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL) [ACL Anthology].
• Keigo Shibata, Kazuki Yano, Ryosuke Takahashi, Wataru Ikeda, and Jun Suzuki. Suppressing Final Layer Hidden State Jumps in Transformer Pretraining. Findings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL) [ACL Anthology].

2025

• Kazuki Yano, Sho Takase, Sosuke Kobayashi, Shun Kiyono, Jun Suzuki. Efficient Construction of Model Family through Progressive Training Using Model Expansion. In Proceedings of Conference on Language Modeling (COLM) [OpenReview].
• Wataru Ikeda, Kazuki Yano, Ryosuke Takahashi, Jaesung Lee, Keigo Shibata, Jun Suzuki. Layer-wise Importance Analysis of Feed-Forward Networks in Transformer-based Language Models. In Proceedings of Conference on Language Modeling (COLM) [OpenReview].
• Kazuki Yano, Takumi Ito, Jun Suzuki. STEP: Staged Parameter-Efficient Pre-training for Large Language Models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL) [ACL Anthology].

2024

• Keito Kudo, Hiroyuki Deguchi, Makoto Morishita, Ryo Fujii, Takumi Ito, Shintaro Ozaki, Koki Natsumi, Kai Sato, Kazuki Yano, Ryosuke Takahashi, Subaru Kimura, Tomomasa Hara, Yusuke Sakai, Jun Suzuki. Document-level Translation with LLM Reranking: Team-J at WMT 2024 General Translation Task. Proceedings of the Ninth Conference on Machine Translation (WMT) [ACL Anthology].