{"id":4162,"date":"2024-03-26T22:18:38","date_gmt":"2024-03-26T23:18:38","guid":{"rendered":"https:\/\/sites.uac.pt\/amendes\/?p=4162"},"modified":"2024-03-26T22:18:39","modified_gmt":"2024-03-26T23:18:39","slug":"look-into-the-machines-mind","status":"publish","type":"post","link":"https:\/\/sites.uac.pt\/amendes\/visualizacao\/look-into-the-machines-mind\/","title":{"rendered":"Look into the machine’s mind"},"content":{"rendered":"\n
<\/span>\"\"
\n

clique no link para visitar a app<\/a><\/p>\n<\/div><\/div>\n\n\n\n

Uma web app capaz de explorar os v\u00e1rios caminhos obtidos da resposta “what is intelligence” do chatGPT<\/p>\n\n\n\n

the data<\/strong>
Using the chatgpt api<\/a>, I ran the same completion prompt “Intelligence is “<\/em> hundreds of times (setting the temperature quite high, at 1.6, for more diverse responses). Given a text, a Large Language Model assigns a probability for the word (token) to come, and it just repeats this process until a completion is\u2026well, complete.<\/p>\n\n\n\n

semantic space<\/strong> (behind)
Each text (a prompt completion or a sub-sequence) has an embedding: a position in a 1536-dimensions space (I call it semantic space, or s\u00b2\u2081\u2085\u2083\u2086). For each response there’s a trajectory through s\u00b2\u2081\u2085\u2083\u2086 that corresponds to each sub-sequence of words, example: “Intelligence is “<\/em> \u2192 “Intelligence is the”<\/em> \u2192 “Intelligence is the ability”<\/em> \u2192 “Intelligence is the ability to”<\/em> \u2192 \u2026 \u2192 full completion<\/em>.<\/p>\n\n\n\n

Because I cannot visualize a 1536-dimensions space (yet), I use a popular technique called Principal Components Analysis<\/a> that tells me, for the set of points I have, what are the most important (principal) dimensions, and allows me to rotate the highly dimensional space so when I look through it, projected into only 3 dimensions, the points are scattered as much as possible. It’s the best (linear)possible reduction of dimensions. In fewer words: it compresses a highly dimensional space into few dimensions while preserving as much info as it can. More or less the same as when for drawing something you choose a perspective (you rotate the object), so it provides the most relevant information. I call this new space s\u00b2\u2083, and it’s what I visualize.<\/p>\n\n\n\n

What you see in the cube is a tree of trajectories that bifurcate. All start with “Intelligence is “<\/em> and progress towards longer and less probable sub-sequences of responses. It’s a different representation of the same tree being visualized on the right (both visualizations communicate).<\/p>\n\n\n\n

The tree visualization<\/strong> (right)
Visualizes all collected completions. It also represents the calculated probability of a word following a text (because the sample is small, this is only a good approximation for the initial levels of the tree), so “Intelligence is the “<\/em> will be followed by “ability”<\/em> ~75% of the times, at 1.6 temperature. If temperature was lower this probability would rise, until achieving certainity at temperature=0.<\/p>\n\n\n\n

By hovering a word, which corresponds to a point in a sub-sequence, you can see in the cube the trajectory from the prompt to all the completions that start with that sub-sequence.<\/p>\n\n\n\n

Try other prompts:<\/strong>
\u00b7 Chatgpt is<\/a>
\u00b7 Best thing about AI is<\/a>
\u00b7 When<\/a>
\u00b7 Santiago Ortiz is<\/a> (yes, this is a selfai. What I found interesting is that it’s ~50% truth ~50% bs, and it feels like it describes alternative versions of my self in the multiverse)
\u00b7 My dream<\/a>
\u00b7 Tell me a story:<\/a>
\u00b7 Intelligence is<\/a><\/p>\n\n\n\n

references<\/strong>
Simulating my friend Philippe<\/a>, where I explain embeddings, and how they are used to run semantic search and to find the proper knowledge from a corpus to use it as context for LLMs prompts
A deeper explanation of LLMs<\/a>, next token prediction, temperature and embeddings, by Stephen Wolfram
English by degrees<\/a> the original Next Word prediction model by Claude Shannon<\/p>\n\n\n\n

moebio<\/a> for more experiments and data proyects<\/p>\n","protected":false},"excerpt":{"rendered":"

Uma web app capaz de explorar os v\u00e1rios caminhos obtidos da resposta “what is intelligence” do chatGPT the dataUsing the chatgpt api, I ran the same completion prompt “Intelligence is “ hundreds of times (setting the temperature quite high, at 1.6, for more diverse responses). Given a text, a Large Language Model assigns a probability for the word (token) […]<\/p>\n","protected":false},"author":159,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[4284,80],"tags":[4286,4285,4287],"class_list":["post-4162","post","type-post","status-publish","format-standard","hentry","category-llms","category-visualizacao","tag-bifurcaes","tag-chatgpt","tag-word-network"],"_links":{"self":[{"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/posts\/4162","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/users\/159"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/comments?post=4162"}],"version-history":[{"count":1,"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/posts\/4162\/revisions"}],"predecessor-version":[{"id":4164,"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/posts\/4162\/revisions\/4164"}],"wp:attachment":[{"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/media?parent=4162"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/categories?post=4162"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.uac.pt\/amendes\/wp-json\/wp\/v2\/tags?post=4162"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}