Заседание научного семинара лаборатории ЛАТАС

Speaker 1: Igor Salnikov (PhD student HSE NN)
Title 1: Efficient fine-tuning of Large Language Models using RBM-Based Localized Learning and Low-Rank Adaptation

The increase in the number of parameters in large language models (LLMs) is accompanied by a significant rise in memory consumption, computational resource requirements, and energy costs — both during the training phase and during inference. A major contributing factor to these limitations is the use of the back propagation algorithm, which necessitates the global propagation of gradients and the storage of intermediate activations at each layer. This report shows an approach to mitigate these costs by combining localized learning based on Restricted Boltzmann Machine (RBM) with the Low-Rank Adaptation (LoRA) method. RBM provides training using localized weight updates that are independent of the global error signal, and can efficiently approximate the distribution of input data without having to save activations for later gradient calculations, which in turn reduces the amount of memory required for fine-tuning the network.

Speaker 2: Vadim Voevodkin (PhD student HSE NN)
Title 2: Application of Large Language Models in Code-Related Tasks: Capabilities, Use Cases, and Challenges in Quality Evaluation
Abstract 2: This talk aims to provide a structured overview of the capabilities of Large Language Models (LLMs) in software development tasks. Particular attention will be given to code search and the generation of comments for source code. The presentation will also address key challenges in evaluating the quality and contextual accuracy of model outputs.

Приглашаются все желающие!
Явка аспирантов школы компьютерных наук обязательна.