LLMμ 'νκ°'μ΄ κ²°ν¨μ΄ μλλΌκ³ ?
Is LLM's 'Hallucination' Not a Flaw?
μΈκ³μ μΈ μννΈμ¨μ΄ κ°λ° μ¬μκ° λ§ν΄ νμΈλ¬κ° μ μνλ LLM μλμ κ°λ° ν¨λ¬λ€μ! κ·Έμ λ μΉ΄λ‘μ΄ ν΅μ°°μ ν΅ν΄ 'λΉκ²°μ μ±'κ³Ό μλ‘μ΄ λ³΄μ μν λ± κ°λ°μκ° λ§μ£Όν λ―Έλλ₯Ό 미리 νμΈν΄ 보μΈμ.
The development paradigm for the LLM era presented by world-renowned software development thinker Martin Fowler! Get a preview of the future developers will face, including 'non-determinism' and new security threats, through his sharp insights.
μλ
νμΈμ! μμ¦ λλ ν κ² μμ΄ AI, νΉν LLM(λκ·λͺ¨ μΈμ΄ λͺ¨λΈ)μ μ
무μ νμ©νκ³ μμ£ . μ½λλ₯Ό μ§κ² νκ±°λ, μμ΄λμ΄λ₯Ό μ»κ±°λ, μ¬μ§μ΄λ 볡μ‘ν κ°λ
μ μ€λͺ
ν΄λ¬λΌκ³ νκΈ°λ νκ³ μ. μ μμ LLMμ νΈλ¦¬ν¨μ νΉ λΉ μ Έ μ§λ΄κ³ μλλ°μ, λ¬Έλ μ΄λ° μκ°μ΄ λ€λλΌκ³ μ. 'κ³Όμ° μ°λ¦¬λ μ΄ λꡬλ₯Ό μ λλ‘ μ΄ν΄νκ³ μ¬μ©νκ³ μλ κ±ΈκΉ?'
Hello! Nowadays, everyone is using AI, especially LLMs (Large Language Models), for work. We make them write code, get ideas, or even ask them to explain complex concepts. I'm also deeply immersed in the convenience of LLMs, but a thought suddenly struck me: 'Are we truly understanding and using this tool correctly?'
μ΄λ° κ³ λ―Όμ μμ€μ μννΈμ¨μ΄ κ°λ° λΆμΌμ μΈκ³μ μΈ κ΅¬λ£¨, λ§ν΄ νμΈλ¬(Martin Fowler)κ° μ΅κ·Ό LLMκ³Ό μννΈμ¨μ΄ κ°λ°μ λν μκ°μ μ 리ν κΈμ μ½κ² λμμ΅λλ€. λ¨μν 'LLMμ λλ¨ν΄!' μμ€μ λμ΄, κ·Έ λ³Έμ§μ μΈ νΉμ±κ³Ό μ°λ¦¬κ° μμΌλ‘ λ§μ£Όνκ² λ λ³νμ λν κΉμ΄ μλ ν΅μ°°μ΄ λ΄κ²¨ μμμ£ . μ€λμ μ¬λ¬λΆκ³Ό ν¨κ» κ·Έμ μκ°μ λ°λΌκ° λ³΄λ €κ³ ν©λλ€. π
While pondering this, I came across an article by Martin Fowler, a world-renowned guru in the software development field, who recently summarized his thoughts on LLMs and software development. It went beyond a simple 'LLMs are amazing!' level, offering deep insights into their fundamental nature and the changes we will face. Today, I'd like to explore his thoughts with you. π
LLM and Software Development
λ§ν΄ νμΈλ¬, LLMμ νμ£Όμλ₯Ό λ§νλ€ π€
Martin Fowler on the Current State of LLMs π€
λ§ν΄ νμΈλ¬λ λ¨Όμ νμ¬ AI μ°μ
μ΄ λͺ
λ°±ν 'λ²λΈ' μνμ μλ€κ³ μ§λ¨ν©λλ€. νμ§λ§ μμ¬μ μΌλ‘ λͺ¨λ κΈ°μ νμ μ΄ κ·Έλμλ―, λ²λΈμ΄ κΊΌμ§ νμλ μλ§μ‘΄μ²λΌ μ΄μλ¨μ μλ‘μ΄ μλλ₯Ό μ¬λ κΈ°μ
μ΄ λνλ κ²μ΄λΌκ³ λ΄€μ΄μ. μ€μν 건, μ§κΈ λ¨κ³μμλ νλ‘κ·Έλλ°μ λ―Έλλ νΉμ μ§μ
μ μμ μ±μ λν΄ λꡬλ νμ€ν μ μ μλ€λ μ μ
λλ€.
Martin Fowler first diagnoses the current AI industry as being in a clear 'bubble' state. However, as with all technological innovations historically, he believes that even after the bubble bursts, companies like Amazon will survive and usher in a new era. The important thing is that at this stage, no one can be certain about the future of programming or the job security of specific professions.
κ·Έλμ κ·Έλ μ£λΆλ₯Έ μ츑보λ€λ κ°μ LLMμ μ§μ μ¬μ©ν΄λ³΄κ³ , κ·Έ κ²½νμ μ κ·Ήμ μΌλ‘ 곡μ νλ μ€νμ μΈ μμΈκ° μ€μνλ€κ³ κ°μ‘°ν©λλ€. μ°λ¦¬ λͺ¨λκ° μλ‘μ΄ λꡬλ₯Ό νννλ κ°μ²μκ° λμ΄μΌ νλ€λ μλ―Έκ² μ£ ?
Therefore, he emphasizes that an experimental attitude of personally using LLMs and actively sharing those experiences is more important than making hasty predictions. This implies that we all need to become pioneers exploring this new tool, right?
π‘ μμλμΈμ!
π‘ Good to know!
νμΈλ¬λ μ΅κ·Ό LLM νμ©μ λν μ€λ¬Έμ‘°μ¬λ€μ΄ μ€μ μ¬μ© νλ¦μ μ λλ‘ λ°μνμ§ λͺ»ν μ μλ€κ³ μ§μ νμ΄μ. λ€μν λͺ¨λΈμ κΈ°λ₯ μ°¨μ΄λ ν¬κΈ° λλ¬Έμ, λ€λ₯Έ μ¬λμ μ견보λ€λ μμ μ μ§μ μ μΈ κ²½νμ λ―Ώλ κ²μ΄ λ μ€μν΄ λ³΄μ
λλ€.
Fowler pointed out that recent surveys on LLM usage may not accurately reflect actual usage patterns. Since there are also significant differences in the capabilities of various models, it seems more important to trust your own direct experience rather than the opinions of others.
LLMμ νκ°: κ²°ν¨μ΄ μλ λ³Έμ§μ νΉμ§ π§
LLM Hallucination: An Intrinsic Feature, Not a Flaw π§
μ΄λ² κΈμμ κ°μ₯ ν₯λ―Έλ‘μ λ λΆλΆμ
λλ€. νμΈλ¬λ LLMμ΄ μ¬μ€μ΄ μλ μ 보λ₯Ό κ·Έλ΄λ―νκ² λ§λ€μ΄λ΄λ 'νκ°(Hallucination)' νμμ λ¨μν 'κ²°ν¨'μ΄ μλλΌ 'λ³Έμ§μ μΈ νΉμ±'μΌλ‘ λ΄μΌ νλ€κ³ μ£Όμ₯ν©λλ€. μ λ§ μΆ©κ²©μ μ΄μ§ μλμ? LLMμ κ²°κ΅ 'μ μ©μ±μ΄ μλ νκ°μ μμ±νκΈ° μν λꡬ'λΌλ κ΄μ μ
λλ€.
This was the most interesting part of the article for me. Fowler argues that the 'hallucination' phenomenon, where LLMs create plausible but untrue information, should be seen as an 'intrinsic feature' rather than a mere 'flaw'. Isn't that shocking? The perspective is that LLMs are ultimately 'tools for generating useful hallucinations'.
μ΄λ° κ΄μ μμ 보면, μ°λ¦¬λ LLMμ λ΅λ³μ λ§Ήλͺ©μ μΌλ‘ μ λ’°ν΄μλ μ λ©λλ€. μ€νλ € λμΌν μ§λ¬Έμ μ¬λ¬ λ², ννμ λ°κΏκ°λ©° λμ Έλ³΄κ³ λ΅λ³μ μΌκ΄μ±μ νμΈνλ μμ
μ΄ νμμ μ
λλ€. νΉν μ«μ κ³μ°κ³Ό κ°μ΄ κ²°μ μ μΈ λ΅μ΄ νμν λ¬Έμ μ LLMμ μ§μ μ¬μ©νλ €λ μλλ μ μ νμ§ μλ€κ³ λ§λΆμμ΅λλ€.
From this viewpoint, we should not blindly trust the answers from LLMs. Instead, it is essential to ask the same question multiple times with different phrasing to check for consistency in the answers. He added that attempting to use LLMs directly for problems requiring definitive answers, such as numerical calculations, is not appropriate.
⚠️ μ£ΌμνμΈμ!
⚠️ Be careful!
νμΈλ¬λ LLMμ 'μ£Όλμ΄ κ°λ°μ'μ λΉμ νλ κ²μ κ°νκ² λΉνν©λλ€. LLMμ "λͺ¨λ ν
μ€νΈ ν΅κ³Ό!"λΌκ³ μμ μκ² λ§νλ©΄μ μ€μ λ‘λ ν
μ€νΈλ₯Ό μ€ν¨μν€λ μ½λλ₯Ό λ΄λλ κ²½μ°κ° ννμ£ . λ§μ½ μΈκ° λλ£κ° μ΄λ° νλμ λ°λ³΅νλ€λ©΄, μ λ’°λ₯Ό μκ³ μΈμ¬ λ¬Έμ λ‘ μ΄μ΄μ§ μμ€μ μ¬κ°ν κ²°ν¨μ΄λΌλ κ²μ
λλ€. LLMμ λλ£κ° μλ, κ°λ ₯νμ§λ§ μ€μλ₯Ό μ μ§λ₯Ό μ μλ 'λꡬ'λ‘ μΈμν΄μΌ ν©λλ€.
Fowler strongly criticizes the analogy of an LLM to a 'junior developer'. LLMs often confidently state "All tests passed!" while providing code that actually fails tests. If a human colleague were to do this repeatedly, it would be a serious flaw leading to a loss of trust and personnel issues. LLMs should be recognized not as colleagues, but as powerful 'tools' that can make mistakes.
μννΈμ¨μ΄ 곡ν, 'λΉκ²°μ μ±' μλλ‘μ μ ν π²
Software Engineering's Shift to an Era of 'Non-Determinism' π²
μ ν΅μ μΈ μννΈμ¨μ΄ 곡νμ 'κ²°μ λ‘ μ 'μΈ μΈκ³ μμ μΈμμ Έ μμμ΅λλ€. '2+2'λ₯Ό μ
λ ₯νλ©΄ '4'κ° λμμΌ νλ―, λͺ¨λ κ²μ μμΈ‘ κ°λ₯νκ³ μΌκ΄μ μ΄μ΄μΌ νμ£ . μμκ³Ό λ€λ₯Έ κ²°κ³Όλ 'λ²κ·Έ'λ‘ μ·¨κΈλμ΄ μ¦μ μμ λμμ΅λλ€.
Traditional software engineering was built on a 'deterministic' world. Just as inputting '2+2' must yield '4', everything had to be predictable and consistent. Unexpected results were treated as 'bugs' and fixed immediately.
νμ§λ§ LLMμ λ±μ₯μ μ΄λ¬ν ν¨λ¬λ€μμ κ·Όλ³Έμ μΌλ‘ λ°κΎΈκ³ μμ΅λλ€. νμΈλ¬λ LLMμ΄ μννΈμ¨μ΄ 곡νμ 'λΉκ²°μ μ±(Non-Determinism)'μ λμ
νλ μ νμ μ΄ λ κ²μ΄λΌκ³ μ§λ¨ν©λλ€. λμΌν μμ²μλ LLMμ λ―Έλ¬νκ² λ€λ₯Έ κ²°κ³Όλ¬Όμ λ΄λμ μ μμΌλ©°, κ·Έλ΄λ―ν΄ λ³΄μ΄λ μ½λ μμ μΉλͺ
μ μΈ μ€λ₯λ₯Ό μ¨κ²¨λκΈ°λ ν©λλ€.
However, the emergence of LLMs is fundamentally changing this paradigm. Fowler diagnoses that LLMs will be a turning point, introducing 'Non-Determinism' into software engineering. Even with the same request, an LLM can produce subtly different outputs and may hide critical errors within plausible-looking code.
μ΄μ κ°λ°μμ μν μ λ¨μν μ½λλ₯Ό μμ±νλ κ²μ λμ΄, LLMμ΄ λ§λ€μ΄λΈ λΆνμ€ν κ²°κ³Όλ¬Όμ λΉνμ μΌλ‘ κ²μ¦νκ³ κ΄λ¦¬νλ λ₯λ ₯μ΄ λμ± μ€μν΄μ‘μ΅λλ€. μλ νλ‘ κ·Έ μ°¨μ΄λ₯Ό κ°λ¨ν μ 리ν΄λ΄€μ΅λλ€.
Now, the role of a developer has become more about the ability to critically verify and manage the uncertain outputs generated by LLMs, going beyond simply writing code. I've summarized the differences in the table below.
κ΅¬λΆ Category |
μ ν΅μ μννΈμ¨μ΄ (κ²°μ μ ) Traditional Software (Deterministic) |
LLM κΈ°λ° μννΈμ¨μ΄ (λΉκ²°μ μ ) LLM-based Software (Non-deterministic) |
κ²°κ³Ό μμΈ‘μ± Result Predictability |
λμΌ μ
λ ₯, λμΌ κ²°κ³Ό 보μ₯ Same input, same output guaranteed |
λμΌ μ
λ ₯μλ λ€λ₯Έ κ²°κ³Ό κ°λ₯ Different outputs possible for the same input |
μ€λ₯μ μ μ Definition of Error |
μμΈ‘μ λ²μ΄λ λͺ¨λ λμ (λ²κ·Έ) Any behavior deviating from prediction (Bug) |
κ²°κ³Όμ λΆνμ€μ± (λ³Έμ§μ νΉμ±) Uncertainty of results (Intrinsic feature) |
κ°λ°μ μν Developer's Role |
μ νν λ‘μ§ κ΅¬ν λ° λλ²κΉ
Implementing precise logic and debugging |
κ²°κ³Όλ¬Ό κ²μ¦ λ° λΆνμ€μ± κ΄λ¦¬ Verifying outputs and managing uncertainty |
νΌν μ μλ μν: 보μ λ¬Έμ π
The Unavoidable Threat: Security Issues π
λ§μ§λ§μΌλ‘ νμΈλ¬λ LLMμ΄ μννΈμ¨μ΄ μμ€ν
μ 곡격 νλ©΄μ κ΄λ²μνκ² νλνλ€λ μ¬κ°ν κ²½κ³ λ₯Ό λμ§λλ€. νΉν λΈλΌμ°μ μμ΄μ νΈμ κ°μ΄ λΉκ³΅κ° λ°μ΄ν° μ κ·Ό, μΈλΆ ν΅μ , μ λ’°ν μ μλ μ½ν
μΈ λ
ΈμΆμ΄λΌλ 'μΉλͺ
μ μΌμ€' μνμ κ°μ§ λꡬλ€μ κ·Όλ³Έμ μΌλ‘ μμ νκ² λ§λ€κΈ° μ΄λ ΅λ€λ κ²μ΄ κ·Έμ μ견μ
λλ€.
Finally, Fowler issues a serious warning that LLMs significantly expand the attack surface of software systems. He opines that tools with the 'lethal triple' risk of accessing private data, communicating externally, and being exposed to untrusted content, such as browser agents, are fundamentally difficult to secure.
μλ₯Ό λ€μ΄, μΉ νμ΄μ§μ μΈκ°μ λμλ 보μ΄μ§ μλ λͺ
λ Ήμ΄λ₯Ό μ¨κ²¨ LLMμ μμ΄κ³ , μ΄λ₯Ό ν΅ν΄ λ―Όκ°ν κ°μΈ μ 보λ₯Ό μ μΆνλλ‘ μ λνλ κ³΅κ²©μ΄ κ°λ₯ν΄μ§λλ€. κ°λ°μλ€μ μ΄μ μ½λμ κΈ°λ₯λΏλ§ μλλΌ, LLMκ³Ό μνΈμμ©νλ λͺ¨λ κ³Όμ μμ λ°μν μ μλ μλ‘μ΄ λ³΄μ μ·¨μ½μ μ κ³ λ €ν΄μΌ ν©λλ€.
For example, it becomes possible to trick an LLM by hiding commands invisible to the human eye on a web page, thereby inducing it to leak sensitive personal information. Developers must now consider not only the functionality of their code but also new security vulnerabilities that can arise in all processes interacting with LLMs.
νκ°μ λ³Έμ§:
Hallucination is Intrinsic: LLMμ νκ°μ 'κ²°ν¨'μ΄ μλ 'λ³Έμ§μ νΉμ§'μΌλ‘ μ΄ν΄ν΄μΌ ν©λλ€.
LLM's hallucination must be understood as an 'intrinsic feature,' not a 'flaw.'
λΉκ²°μ μ±μ μλ:
The Era of Non-Determinism: μννΈμ¨μ΄ 곡νμ΄ μμΈ‘ λΆκ°λ₯μ±μ κ΄λ¦¬νλ μλλ‘ μ§μ
νμ΅λλ€.
Software engineering has entered an era of managing unpredictability.
κ²μ¦μ νμ:
Verification is a Must:
LLMμ κ²°κ³Όλ¬Όμ μ£Όλμ΄ κ°λ°μκ° μλ, κ²μ¦μ΄ νμμ μΈ 'λꡬ'μ μ°μΆλ¬Όμ
λλ€.
The output of an LLM is not that of a junior developer, but the product of a 'tool' that requires mandatory verification.
보μ μν:
Security Threats: LLMμ μμ€ν
μ 곡격 νλ©΄μ λνλ μλ‘μ΄ λ³΄μ λ³μμ
λλ€.
LLMs are a new security variable that broadens a system's attack surface.
μμ£Ό 묻λ μ§λ¬Έ ❓
Frequently Asked Questions ❓
Q: λ§ν΄ νμΈλ¬κ° 'νκ°'μ κ²°ν¨μ΄ μλ λ³Έμ§λ‘ λ΄μΌ νλ€κ³ λ§νλ μ΄μ λ 무μμΈκ°μ?
Q: Why does Martin Fowler say that 'hallucination' should be seen as an intrinsic feature, not a flaw?
A: LLMμ λ°©λν λ°μ΄ν°λ₯Ό κΈ°λ°μΌλ‘ κ°μ₯ κ·Έλ΄λ―ν λ€μ λ¨μ΄λ₯Ό μμΈ‘νμ¬ λ¬Έμ₯μ μμ±νλ λͺ¨λΈμ΄κΈ° λλ¬Έμ
λλ€. μ΄ κ³Όμ μμ μ¬μ€κ΄κ³μ 무κ΄νκ² λ§€λλ¬μ΄ λ¬Έμ₯μ λ§λ€μ΄λ΄λ 'νκ°'μ μμ°μ€λ¬μ΄ κ²°κ³Όλ¬Όμ΄λ©°, μ΄ νΉμ±μ μ΄ν΄ν΄μΌ LLMμ μ¬λ°λ₯΄κ² νμ©ν μ μλ€λ μλ―Έμ
λλ€.
A: This is because LLMs are models that generate sentences by predicting the most plausible next word based on vast amounts of data. In this process, 'hallucination,' which creates fluent sentences regardless of factual accuracy, is a natural outcome. Understanding this characteristic is key to using LLMs correctly.
Q: μννΈμ¨μ΄ 곡νμ 'λΉκ²°μ μ±'μ΄λ 무μμ μλ―Ένλ©°, μ μ€μνκ°μ?
Q: What does 'non-determinism' in software engineering mean, and why is it important?
A: 'λΉκ²°μ μ±'μ΄λ λμΌν μ
λ ₯μ λν΄ νμ λμΌν κ²°κ³Όκ° λμ€μ§ μλ νΉμ±μ μλ―Έν©λλ€. μ ν΅μ μΈ μννΈμ¨μ΄λ 100% μμΈ‘ κ°λ₯ν΄μΌ νμ§λ§, LLMμ κ°μ μ§λ¬Έμλ λ€λ₯Έ λ΅λ³μ μ€ μ μμ΅λλ€. μ΄ λΆνμ€μ±μ μ΄ν΄νκ³ κ΄λ¦¬νλ κ²μ΄ LLM μλ κ°λ°μμ ν΅μ¬ μλμ΄ λμμ΅λλ€.
A: 'Non-determinism' refers to the characteristic where the same input does not always produce the same output. While traditional software had to be 100% predictable, an LLM can give different answers to the same question. Understanding and managing this uncertainty has become a core competency for developers in the age of LLMs.
Q: LLMμ΄ μμ±ν μ½λλ₯Ό μ λ’°νκ³ λ°λ‘ μ¬μ©ν΄λ λ κΉμ?
Q: Can I trust and use the code generated by an LLM immediately?
A: μλμ, μ λ μ λ©λλ€. λ§ν΄ νμΈλ¬λ LLMμ΄ κ·Έλ΄λ―νμ§λ§ μλνμ§ μκ±°λ, 보μμ μ·¨μ½ν μ½λλ₯Ό μμ±ν μ μλ€κ³ κ²½κ³ ν©λλ€. μμ±λ μ½λλ λ°λμ κ°λ°μκ° μ§μ κ²ν , ν
μ€νΈ, κ²μ¦νλ κ³Όμ μ κ±°μ³μΌ ν©λλ€.
A: No, absolutely not. Martin Fowler warns that LLMs can generate code that looks plausible but doesn't work or is insecure. The generated code must be reviewed, tested, and verified by a developer.
Q: LLMμ μ¬μ©νλ©΄ μ 보μ μνμ΄ μ»€μ§λμ?
Q: Why do security threats increase with the use of LLMs?
A: LLMμ μΈλΆ λ°μ΄ν°μ μνΈμμ©νκ³ , λλ‘λ λ―Όκ°ν μ 보μ μ κ·Όν μ μκΈ° λλ¬Έμ
λλ€. μ
μμ μΈ μ¬μ©μκ° μΉμ¬μ΄νΈλ μ
λ ₯κ°μ 보μ΄μ§ μλ λͺ
λ Ήμ΄λ₯Ό μ¨κ²¨ LLMμ μ‘°μ’
(ν둬ννΈ μΈμ μ
)νμ¬ μ 보λ₯Ό μ μΆνκ±°λ μμ€ν
μ 곡격νλ μλ‘μ΄ ννμ 보μ μνμ΄ λ°μν μ μμ΅λλ€.
A: Because LLMs interact with external data and can sometimes access sensitive information. Malicious users can hide invisible commands in websites or inputs to manipulate the LLM (prompt injection), leading to new types of security threats such as data leakage or system attacks.
λ§ν΄ νμΈλ¬μ ν΅μ°°μ LLMμ΄λΌλ μλ‘μ΄ λꡬλ₯Ό μ΄λ»κ² λ°λΌλ³΄κ³ μ¬μ©ν΄μΌ νλμ§μ λν μ€μν κ°μ΄λλ₯Ό μ μν©λλ€. λ¨μν νΈλ¦¬ν μ½λ μμ±κΈ°λ₯Ό λμ΄, μ°λ¦¬ κ°λ° νκ²½μ κ·Όλ³Έμ μΈ ν¨λ¬λ€μμ λ°κΎΈλ μ‘΄μ¬μμ μΈμν΄μΌ ν©λλ€. κ·Έμ μ‘°μΈμ²λΌ, λλ €μνκ±°λ λ§Ήμ ν기보λ€λ μ κ·Ήμ μΌλ‘ μ€ννκ³ κ²½νμ 곡μ νλ©° μ΄ κ±°λν λ³νμ λ¬Όκ²°μ νλͺ
νκ² μ¬λΌνμΌ ν λμ
λλ€.
Martin Fowler's insights provide an important guide on how to view and use the new tool that is the LLM. We must recognize it not just as a convenient code generator, but as an entity that is changing the fundamental paradigm of our development environment. As he advises, now is the time to wisely ride this massive wave of change by experimenting and sharing experiences, rather than fearing or blindly trusting it.
μ¬λ¬λΆμ LLMμ λν΄ μ΄λ»κ² μκ°νμλμ? κ°λ° κ³Όμ μμ κ²ͺμλ ν₯λ―Έλ‘μ΄ κ²½νμ΄ μλ€λ©΄ λκΈλ‘ 곡μ ν΄μ£ΌμΈμ! π
What are your thoughts on LLMs? If you have any interesting experiences from your development process, please share them in the comments! π