본문 바로가기

인공지능

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning https://arxiv.org/abs/2405.12130 MoRA: High-Rank Updating for Parameter-Efficient Fine-TuningLow-rank adaptation is a popular parameter-efficient fine-tuning method for large language models. In this paper, we analyze the impact of low-rank updating, as implemented in LoRA. Our findings suggest that the low-rank updating mechanism may limit the abarxiv.org 초록Low-rank adaptation (LoRA)는 대형 언어 모델(.. 더보기
ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuning https://arxiv.org/abs/2411.05003 ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-TuningRecently, breakthroughs in video modeling have allowed for controllable camera trajectories in generated videos. However, these methods cannot be directly applied to user-provided videos that are not generated by a video model. In this paper, we present Rearxiv.org .. 더보기
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT https://arxiv.org/abs/1909.05840 Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTTransformer based architectures have become de-facto models used for a range of Natural Language Processing tasks. In particular, the BERT based models achieved significant accuracy gain for GLUE tasks, CoNLL-03 and SQuAD. However, BERT based models have aarxiv.org 초록Transformer 기반 아키텍처는 다양한 자연어 처리 작업에.. 더보기
MambaStock: Selective state space model for stock prediction 회사 프로젝트가 바빠서 너무 못했다. 이제 나왔으니, 다시 인공지능을 해야지 https://arxiv.org/abs/2402.18959 MambaStock: Selective state space model for stock predictionThe stock market plays a pivotal role in economic development, yet its intricate volatility poses challenges for investors. Consequently, research and accurate predictions of stock price movements are crucial for mitigating risks. Traditional time series marxiv... 더보기
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions 프로젝트 때문에, 너무 바빠서 잘 못하고 있다.... 오랜만에 하나... https://arxiv.org/abs/2102.12122 Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without ConvolutionsAlthough using convolutional neural networks (CNNs) as backbones achieves great successes in computer vision, this work investigates a simple backbone network useful for many dense prediction tasks without convolutions. Unlike the rec.. 더보기
VoiceLDM: Text-to-Speech with Environmental Context https://arxiv.org/abs/2309.13664 VoiceLDM: Text-to-Speech with Environmental ContextThis paper presents VoiceLDM, a model designed to produce audio that accurately follows two distinct natural language text prompts: the description prompt and the content prompt. The former provides information about the overall environmental context of tharxiv.org 초록본 논문에서는 두 개의 서로 다른 자연어 텍스트 프롬프트인 설명 프롬프트와 내용 프.. 더보기
DeepNet: Scaling Transformers to 1,000 Layers (부록 추가 필요) https://arxiv.org/abs/2203.00555 DeepNet: Scaling Transformers to 1,000 LayersIn this paper, we propose a simple yet effective method to stabilize extremely deep Transformers. Specifically, we introduce a new normalization function (DeepNorm) to modify the residual connection in Transformer, accompanying with theoretically derived iarxiv.org 이 논문에서는 매우 깊은 트랜스포머(Transformers)를 안정화시키는 간단하면서도 효과적인 .. 더보기
Scaling Vision Transformers (부록 추가 필요) https://arxiv.org/abs/2106.04560 Scaling Vision TransformersAttention-based neural networks such as the Vision Transformer (ViT) have recently attained state-of-the-art results on many computer vision benchmarks. Scale is a primary ingredient in attaining excellent results, therefore, understanding a model's scalinarxiv.org 초록Vision Transformer(ViT)와 같은 어텐션 기반 신경망은 최근 여러 컴퓨터 비전 벤치마크에서 최첨단 결과를 달성.. 더보기

티스토리툴바