Tag: pretrain

News
Disclosing OpenAI GPT-4's vision+text model, data, and cost to train (speculated)
Tutorials

OpenAI GPT-4์˜ ๋น„์ „+ํ…์ŠคํŠธ ๋ชจ๋ธ, ๋ฐ์ดํ„ฐ ๋ฐ ํ•™์Šต ๋น„์šฉ ๊ณต๊ฐœ (์ถ”์ธก)

OpenAI๋Š” GPT-4์˜ ์•„ํ‚คํ…์ฒ˜, ํŒŒ๋ผ๋ฏธํ„ฐ ์ˆ˜, ํ•™์Šต ๋ฐ์ดํ„ฐ ๋˜๋Š” ์ปดํ“จํŒ… ์˜ˆ์‚ฐ์„ ๊ณต๊ฐœํ•˜์ง€ ์•Š๊ธฐ๋กœ ์•…๋ช…๋†’์œผ๋ฉฐ, ์ด๋Š” ๊ธฐ์ˆ  ๋ณด๊ณ ์„œ๋ฅผ ML ์ปค๋ฎค๋‹ˆํ‹ฐ๋ฅผ ์œ„ํ•œ ๋กœ๋ฅด์ƒคํ ํ…Œ์ŠคํŠธ๋กœ ๋ฐ”๊พธ์—ˆ์Šต๋‹ˆ๋‹ค.

4์›” 1, 2023ยท 1 min read
A Review of GPT-4's Technical Report (GPT-4 in a Nutshell)
Tutorials

GPT-4์˜ ๊ธฐ์ˆ  ๋ณด๊ณ ์„œ ๋ฆฌ๋ทฐ (ํ•œ๋ˆˆ์— ๋ณด๋Š” GPT-4)

GPT-4์˜ ๊ธฐ์ˆ  ๋ณด๊ณ ์„œ๋Š” ์•„ํ‚คํ…์ฒ˜ ์„ธ๋ถ€์‚ฌํ•ญ์ด ๋ถ€์กฑํ•˜๊ณ  ๋Šฅ๋ ฅ ์ฃผ์žฅ์ด ๋งŽ๊ธฐ๋กœ ์•…๋ช…๋†’์œผ๋ฉฐ, ์ด๋Š” ๋ช…ํ™•ํ•œ ๋ฆฌ๋ทฐ๋ฅผ ๊ทธ ์œ„์— ๊ตฌ์ถ•ํ•˜๋Š” ๋ชจ๋“  ์‚ฌ๋žŒ์„ ์œ„ํ•ด ํ•„์ˆ˜ ์ฝ๊ธฐ๋กœ ๋งŒ๋“ญ๋‹ˆ๋‹ค.

3์›” 25, 2023ยท 1 min read
In-depth review of OpenAI's GPT-3 : Language Models are Few-Shot Learners (Part 3/3: Results&Rest)
Tutorials

OpenAI์˜ GPT-3์— ๋Œ€ํ•œ ์‹ฌ์ธต ๋ฆฌ๋ทฐ : ์–ธ์–ด ๋ชจ๋ธ์€ ์†Œ์ˆ˜ ์ƒท ํ•™์Šต์ž์ž…๋‹ˆ๋‹ค (ํŒŒํŠธ 3/3: ๊ฒฐ๊ณผ ๋ฐ ๋‚˜๋จธ์ง€)

์ด 3๋ถ€์ž‘ GPT-3 ์‹ฌ์ธต ๋ถ„์„์˜ ์ตœ์ข… ํšŒ์ฐจ๋Š” ๊ฒฐ๊ณผ์— ๋„๋‹ฌํ•ฉ๋‹ˆ๋‹ค: 1,750์–ต ๊ฐœ์˜ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ๋ฒค์น˜๋งˆํฌ ์Šค์œ„ํŠธ์— ํˆฌ์ž…ํ•˜๊ณ  ๋ฏธ์„ธ ์กฐ์ •์„ ์™„์ „ํžˆ ๊ฑด๋„ˆ๋›ธ ๋•Œ ์‹ค์ œ๋กœ ์–ด๋–ค ์ผ์ด ๋ฐœ์ƒํ•˜๋Š”์ง€์— ๋Œ€ํ•ด.

3์›” 8, 2023ยท 1 min read
In-depth review of OpenAI's GPT-3 : Language Models are Few-Shot Learners (Part 2/3: Results)
Tutorials

OpenAI์˜ GPT-3์— ๋Œ€ํ•œ ์‹ฌ์ธต ๊ฒ€ํ† : ์–ธ์–ด ๋ชจ๋ธ์€ Few-Shot ํ•™์Šต์ž์ž…๋‹ˆ๋‹ค (ํŒŒํŠธ 2/3: ๊ฒฐ๊ณผ)

์ด GPT-3 ์‹ฌ์ธต ๋ถ„์„์˜ 2๋ถ€๋Š” ์„ค์ •์„ ๋„˜์–ด ์‹ค์ œ๋กœ ์‚ฌ๋žŒ๋“ค์ด NLP๋ฅผ ์žฌ์ƒ๊ฐํ•˜๊ฒŒ ํ•œ ๊ฒฐ๊ณผ๋กœ ์ด๋™ํ•ฉ๋‹ˆ๋‹ค.

2์›” 25, 2023ยท 1 min read
ChatGPT/ChatGPT Plus/InstructGPT:Training language models to follow instructions with human feedback
Tutorials

ChatGPT/ChatGPT Plus/InstructGPT: ์ธ๊ฐ„ ํ”ผ๋“œ๋ฐฑ์œผ๋กœ ์ง€์‹œ๋ฅผ ๋”ฐ๋ฅด๋„๋ก ์–ธ์–ด ๋ชจ๋ธ ํ›ˆ๋ จ

ChatGPT ์ œํ’ˆ ๋’ค์—๋Š” InstructGPT ๋…ผ๋ฌธ์ด ์žˆ์œผ๋ฉฐ, ์ด ์ƒ์„ธ ๋ฆฌ๋ทฐ๋Š” ์›์‹œ GPT-3์„ ์‚ฌ๋žŒ๋“ค์ด ์‹ค์ œ๋กœ ๋Œ€ํ™”ํ•˜๊ณ  ์‹ถ์–ด ํ•˜๋Š” ๊ฒƒ์œผ๋กœ ๋ณ€ํ™˜ํ•œ ํŒŒ์ดํ”„๋ผ์ธ์„ ์ž์„ธํžˆ ๋ถ„์„ํ•ฉ๋‹ˆ๋‹ค.

1์›” 28, 2023ยท 1 min read
Explain how ChatGPT/ChatGPT Plus works in 3 minutes
Tutorials

3๋ถ„ ์•ˆ์— ChatGPT/ChatGPT Plus๊ฐ€ ์–ด๋–ป๊ฒŒ ์ž‘๋™ํ•˜๋Š”์ง€ ์„ค๋ช…

ChatGPT๋Š” ์™ธ๋ถ€์—์„œ๋Š” ๋งˆ๋ฒ•์ฒ˜๋Ÿผ ๋ณด์ด์ง€๋งŒ, ์–ด๋””๋ฅผ ๋ด์•ผ ํ•˜๋Š”์ง€ ์•Œ๋ฉด ๋ ˆ์‹œํ”ผ๋Š” ์ž˜ ๋ฌธ์„œํ™”๋˜์–ด ์žˆ์œผ๋ฉฐ, ์ด 3๋ถ„ ์„ค๋ช…์€ ํ•ต์‹ฌ ์•„์ด๋””์–ด๋ฅผ ์š”์•ฝํ•ฉ๋‹ˆ๋‹ค.

1์›” 28, 2023ยท 1 min read
[Olewave's Review] CLIP (3/3): Learning Transferable Visual Models From Natural Language Supervision
Tutorials

[Olewave's Review] CLIP (3/3): ์ž์—ฐ์–ด ๊ฐ๋…์œผ๋กœ๋ถ€ํ„ฐ์˜ ์ „์ด ๊ฐ€๋Šฅํ•œ ์‹œ๊ฐ ๋ชจ๋ธ ํ•™์Šต

์ด CLIP ๊ฒ€ํ† ์˜ ๋งˆ์ง€๋ง‰ ๋ถ€๋ถ„์€ ๊ฒฐ๊ณผ ์„น์…˜์— ๋„๋‹ฌํ•˜๋ฉฐ, ์—ฌ๊ธฐ์„œ OpenAI์˜ ๋Œ€์กฐ์  ์ด๋ฏธ์ง€-ํ…์ŠคํŠธ ๋ชจ๋ธ์ด ํฅ๋ฏธ๋กœ์šด ์•„์ด๋””์–ด์—์„œ ์ „์ฒด ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ ์Šคํƒ์˜ ๊ธฐ์ดˆ ๊ณ„์ธต ๊ธฐ๋ณธ ์š”์†Œ๋กœ ์ „ํ™˜๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

1์›” 14, 2023ยท 1 min read
[Olewave's Review] CLIP (2/3): Learning Transferable Visual Models From Natural Language Supervision
Tutorials

[Olewave's Review] CLIP (2/3): ์ž์—ฐ์–ด ๊ฐ๋…์œผ๋กœ๋ถ€ํ„ฐ์˜ ์ „์ด ๊ฐ€๋Šฅํ•œ ์‹œ๊ฐ ๋ชจ๋ธ ํ•™์Šต

OpenAI์˜ CLIP์€ ๊ณ ์ • ๋ ˆ์ด๋ธ” ์„ธํŠธ๋ฅผ ๋ฒ„๋ฆฌ๊ณ  ๋Œ€์‹  ์ธํ„ฐ๋„ท์—์„œ ์ˆ˜์ง‘ํ•œ 4์–ต ๊ฐœ์˜ (์ด๋ฏธ์ง€, ํ…์ŠคํŠธ) ์Œ์œผ๋กœ ํ›ˆ๋ จํ•จ์œผ๋กœ์จ ์ปดํ“จํ„ฐ ๋น„์ „์˜ ์Šคํฌ๋ฆฝํŠธ๋ฅผ ๋’ค์ง‘์—ˆ์Šต๋‹ˆ๋‹ค.

1์›” 1, 2023ยท 1 min read
[Olewave's Review] CLIP (1/3): Learning Transferable Visual Models From Natural Language Supervision
Tutorials

[Olewave's Review] CLIP (1/3): ์ž์—ฐ์–ด ๊ฐ๋…์œผ๋กœ๋ถ€ํ„ฐ์˜ ์ „์ด ๊ฐ€๋Šฅํ•œ ์‹œ๊ฐ ๋ชจ๋ธ ํ•™์Šต

BLIP, LLaVA, ๋˜๋Š” ๊ฐ€์น˜ ์žˆ๋Š” ์Œ์„ฑ-LLM์ด ์žˆ๊ธฐ ์ „์— CLIP์ด ์žˆ์—ˆ์œผ๋ฉฐ, ์ด๊ฒƒ์ด ์ด์•ผ๊ธฐ๊ฐ€ ์‹œ์ž‘๋˜๋Š” ๊ณณ์ž…๋‹ˆ๋‹ค.

12์›” 17, 2022ยท 1 min read