AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing

What it is about

AuthorMix trains one LoRA adapter for each of about ten high-resource authors. For a new low-resource target, it mixes these adapters with scalar weights, learned per layer with reinforcement learning (GRPO), using the joint style and meaning score of a few target examples as reward. The base model and adapters stay frozen; only about 128 weights are learned for four adapters, taking roughly forty minutes on one GPU. The best joint score is about 0.34, against 0.29 to 0.30 for a single monolithic model trained on the same data, and its meaning score is clearly higher. Layer-wise weights concentrate in the middle layers, and the mixing weights can be negative. Gradient-free search (LoRAHub) found little meaningful weight. Human evaluation ranks the method best or tied on every dimension.

Ideas

• I1. Deltas as a library. Treat each author delta as a sparse set of overrides and allow a target to be a signed mixture of at most k deltas, with k≤4. Learn the weights from the translator's residual rather than from a neural reward, keeping the choice of deltas in BTreeMap order.
• I2. Per-layer residual attribution. The translator's exact mode already reports bits per layer (reader, encoder, stack). Reporting which layer a delta buys bits in is the counted analogue of the paper's middle-layer finding.
• I3. Negative overrides. Allow an override to remove a standard form (for example a KJV archaism when the target is modern) as well as to add one. Test on WEB and ASV pairs.
• I4. Exact search instead of gradient-free search. The paper reports that gradient-free search failed. Mova can search over on/off choices of a few overrides exhaustively, which is feasible at small k and deterministic.

For Mova

This is the closest existing design to the author DELTA of tools/translate: a standard plus sparse, learned, weighted overrides. The training method (GRPO) is not for Mova; the mixture form, the per-layer report and the negative weights are. Run each variant twice and report it against the current delta before any claim. Relates to open problems 6 and 8.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work. arXiv 2603.23069

@misc{thillainathan2026authormix,
  title = {{AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing}},
  author = {Thillainathan, Sarubi and Lee, Ji-Ung and Sullivan, Michael and Koller, Alexander},
  year = {2026},
  eprint = {2603.23069},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.23069}
}

References

37 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marwa Abdulhai, Isadora White, Yanming Wan, Ibrahim Qureshi, Joel Z. Leibo, Max Kleiman-Weiner, and Natasha Jaques. 2026. How LLMs Distort Our Written Language. arXiv: 2603.18161 [cs.CL]. ↗
2. Nikolay Babakov, David Dale, Varvara Logacheva, and Alexander Panchenko. 2022. A large-scale computa tional study of content preservation measures for text style transfer and paraphrase generation. In Proceed ings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 300–321, Dublin, Ireland. find
3. EMNLP 2024, pages 13376–13390, Miami, Florida, USA. find
4. Alexandra Chronopoulou, Matthew Peters, Alexander Fraser, and Jesse Dodge. 2023. AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2054–2063, Dubrovnik, Croatia. find
5. Edward J Hu, shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. find
6. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What Does BERT Look at? An Analysis of BERT’s Attention. In Pro ceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276–286, Florence, Italy. find
7. Mohsen Fayyaz, Ehsan Aghazadeh, Ali Modarressi, Hosein Mohebbi, and Mohammad Taher Pilehvar. 2021. Not All Models Localize Linguistic Knowl edge in the Same Place: A Layer-wise Probing on BERToids’ Representations. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 375– 388, Punta Cana, Dominican Republic. find
8. Jillian Fisher, Skyler Hallinan, Ximing Lu, Mitchell L Gordon, Zaid Harchaoui, and Yejin Choi. 2024. StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements. In Proceedings of the 2024 Conference on Empiri cal Methods in Natural Language Processing, pages 4172–4206, Miami, Florida, USA. find our card
9. Moemen Gaafar, Anurag Sarkar, and Matthew Guzdial. 2025. Few-shot Style-Conditioned LLM Text Gener ation via Latent Interpolation. find
10. Martin Gerlach and Francesc Font-Clos. 2018. A stan dardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguis tics. arXiv: 1812.08092 [cs.CL]. ↗
11. Gurobi Optimization, LLC. 2026. Gurobi Optimizer Reference Manual. find
12. Jiaqi Han, Jingwen Ye, Shunyu Liu, Haofei Zhang, Jie Song, Zunlei Feng, and Mingli Song. 2025. Rein forced Model Merging. In 2025 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. find
13. Carolin Holtermann, Markus Frohmann, Navid Rekab saz, and Anne Lauscher. 2024. What the Weight?! A Unified Framework for Zero-Shot Knowledge Com position. In Findings of the Association for Compu tational Linguistics: EACL 2024, pages 1138–1157, St. Julian’s, Malta. find
14. Zachary Horvitz, Ajay Patel, Kanishk Singh, Chris Callison-Burch, Kathleen McKeown, and Zhou Yu. 2024. TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings. In Findings of the Association for Computational Linguistics: Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. 2024. LoraHub: Effi cient Cross-Task Generalization via Dynamic LoRA Composition. In First Conference on Language Mod eling. find
15. Javier Huertas-Tato, Alejandro Martín, and David Ca macho. 2024. Understanding writing style in social media with a supervised contrastively pre-trained transformer. Knowledge-Based Systems 296:111867. find
16. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Worts man, Suchin Gururangan, Ludwig Schmidt, Han naneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In The Eleventh Inter national Conference on Learning Representations . find
17. Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep Learning for Text Style Transfer: A Survey. Computational Linguistics 48(1):155–205. find
18. Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020. Reformulating Unsupervised Style Transfer as Para phrase Generation. In Proceedings of the 2020 Con ference on Empirical Methods in Natural Language Processing (EMNLP), pages 737–762, Online. find
19. Shuai Liu, Shantanu Agarwal, and Jonathan May. 2024a. Authorship Style Transfer with Policy Opti mization. arXiv: 2403.08043 [cs.CL]. ↗
20. Xinyue Liu, Harshita Diddee, and Daphne Ippolito. 2024b. Customizing Large Language Model Gener ation Style using Parameter-Efficient Finetuning. In Proceedings of the 17th International Natural Language Generation Conference, pages 412–426, Tokyo, Japan. find
21. Ajay Patel, Nicholas Andrews, and Chris Callison-Burch. 2024. Low-Resource Authorship Style Trans fer: Can Non-Famous Authors Be Imitated? arXiv: 2212.08986 [cs.CL]. ↗ our card
22. Ajay Patel, Jiacheng Zhu, Justin Qiu, Zachary Horvitz, Marianna Apidianaki, Kathleen McKeown, and Chris Callison-Burch. 2025. StyleDistance: Stronger Content-Independent Style Embeddings with Syn thetic Parallel Examples. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa pers), pages 8662–8685, Albuquerque, New Mexico. find
23. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. AdapterFusion: Non-Destructive Task Composition for Transfer Learning. In Proceedings of the 16th Conference of the European Chapter of the Associ ation for Computational Linguistics: Main Volume, pages 487–503, Online. find
24. Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, and Samy Jelassi. 2025. LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks. In Proceedings of the 31st Inter national Conference on Computational Linguistics: Industry Track, pages 644–655, Abu Dhabi, UAE. find
25. J. Rapin and O. Teytaud. 2018. Nevergrad - A gradient-free optimization platform. https://GitHub.com/Face bookResearch/Nevergrad. ↗
26. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat ural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. find
27. Parker Riley, Noah Constant, Mandy Guo, Girish Kumar, David Uthus, and Zarana Parekh. 2021. TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling. In Proceedings of the 59th Annual Meeting of the Association for Compu tational Linguistics and the 11th International Joint Conference on Natural Language Processing (Vol ume 1: Long Papers), pages 3786–3800, Online. find
28. Rafael A. Rivera-Soto, Olivia Elizabeth Miano, Juanita Ordonez, Barry Y. Chen, Aleem Khan, Marcus Bishop, and Nicholas Andrews. 2021. Learning Uni versal Authorship Representations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 913–919, On line and Punta Cana, Dominican Republic. find
29. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathemati cal Reasoning in Open Language Models. arXiv: 2402.03300 [cs.CL]. ↗
30. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT Rediscovers the Classical NLP Pipeline. In Proceedings of the 57th Annual Meeting of the Asso ciation for Computational Linguistics, pages 4593– 4601, Florence, Italy. find
31. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. Neural Network Acceptability Judg ments. Transactions of the Association for Computa tional Linguistics 7:625–641. find
32. Anna Wegmann, Marijn Schraagen, and Dong Nguyen. 2022. Same Author or Just Same Topic? Towards Content-Independent Style Representations. In Pro ceedings of the 7th Workshop on Representation Learning for NLP, pages 249–268, Dublin, Ireland. find
33. Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for Style. In Proceedings of COLING 2012, pages 2899–2914, Mumbai, India. find our card
34. Yiyan Xu, Jinghao Zhang, Alireza Salemi, Xinting Hu, Wenjie Wang, Fuli Feng, Hamed Zamani, Xiangnan He, and Tat-Seng Chua. 2025. Personalized Gener ation In Large Model Era: A Survey. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24607–24649, Vienna, Austria. find
35. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. TIES-Merging: Resolving Interference When Merging Models. In Thirty-seventh Conference on Neural Information Processing Systems. find
36. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. 2024. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. In Proceedings of the 62nd Annual Meeting of the As sociation for Computational Linguistics (Volume 3: System Demonstrations), pages 400–410, Bangkok, Thailand. find
37. Yichu Zhou and Vivek Srikumar. 2022. A Closer Look at How Fine-tuning Changes BERT. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1046–1061, Dublin, Ireland. Measurement Away from source style (STAR) Toward target style (STAR) MIS (NLI bidirectional) CoLA fluency Value 0.91 0.01 0.79 0.89 find

This page in Markdown → https://md.mova.elamur.ai/free/en/docs/papers/thillainathan-2026-authormix-modular-authorship-style-transfer/