Large language models (LLMs) and their fine-tuning techniques have demonstrated superior performance in various language understanding and generation tasks.This paper explores fine-tuning LLMs for predicting stock returns with financial newsflow.Return prediction is fundamental for subsequent tasks like portfolio construction and optimization in quantitative investing. We formulate the model to include a text representation and forecasting modules. We propose to compare the encoder-only and decoder-only LLMs, considering they generate text representations in distinct ways.The impact of these different representations on return forecasting remains an open question.Meanwhile, we compare two simple methods of integrating LLMs’ token-level representations into the forecasting module.The experiments on real investment universes reveal that:(1) aggregated representations from LLMs’ token-level embeddings generally produce return predictions that enhance the performance of long-only and long-short portfolios;(2) in the relatively large investment universe, the decoder LLMs-based prediction model leads to stronger portfolios, whereas in the small universes, there are no consistent winners;(3) return predictions derived from LLMs’ text representations are a strong signal for portfolio construction, outperforming conventional sentiment scores.These findings shed light on developing suitable LLM fine-tuning methods for return prediction-based portfolio construction.