{"id":51946,"date":"2020-07-23T18:15:00","date_gmt":"2020-07-23T18:15:00","guid":{"rendered":"https:\/\/icrowdnewswire.com\/?p=2669371"},"modified":"2020-07-23T18:15:00","modified_gmt":"2020-07-23T18:15:00","slug":"openais-new-gpt-3-language-explained-in-under-3-minutes","status":"publish","type":"post","link":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/","title":{"rendered":"OpenAI\u2019s new GPT-3 language explained in under 3 minutes"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" width=\"233\" height=\"24\" src=\"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png\" class=\"webfeedsFeaturedVisual wp-post-image\" alt=\"\" style=\"display: block; margin-bottom: 5px; clear:both;max-width: 100%;\" link_thumbnail=\"\" \/><\/p>\n<header class=\"c-post-header\">\n<p class=\"c-post-subtitle\">Peek under the hood of GPT-3 in under 3 minutes<\/p>\n<\/header>\n<div class=\"c-split\">\n<div class=\"u-wrapper-bleed\">\n<div class=\"c-post-image\">\n<header class=\"c-post-header\"><\/header>\n<div class=\"c-split\">\n<div class=\"u-wrapper-bleed\">\n<div class=\"c-post-image\"><img loading=\"lazy\" decoding=\"async\" class=\"is-passedTop is-inView\" src=\"https:\/\/img-cdn.tnwcdn.com\/image?fit=1280%2C720&amp;url=https%3A%2F%2Fcdn0.tnwcdn.com%2Fwp-content%2Fblogs.dir%2F1%2Ffiles%2F2020%2F07%2FUntitled-design-1-9.png&amp;signature=a34693d53c2af98bd02cd5dc6c778a7a\" alt=\"Untitled design (1)\" width=\"683\" height=\"384\" data-offset=\"631.25\" data-lazy-loaded=\"true\" \/><a class=\"c-post-image-credit\" href=\"http:\/\/pluspng.com\/png-stopwatch-3425.html\">U<\/a><\/div>\n<\/div>\n<div class=\"ad c-post-ad c-post-ad--top\">\n<div id=\"TNW_ROS_TOP_1307465\" class=\"ad-unit ad-unit\" data-ad=\"TNW_ROS_TOP\" data-ad-initialized=\"true\" data-google-query-id=\"CLT5oLLP4-oCFcrcGwodou4OhQ\">\n<p>So, you&rsquo;ve seen some amazing GPT-3 demos on Twitter (if not, where have you been?). This mega machine learning model, created by&nbsp;<a href=\"https:\/\/openai.com\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">OpenAI<\/a>, can write it&rsquo;s own op-eds, poems, articles, and even working code:<\/p>\n<div class=\"twitter-tweet twitter-tweet-rendered\"><\/div>\n<div class=\"twitter-tweet twitter-tweet-rendered\"><\/div>\n<div class=\"twitter-tweet twitter-tweet-rendered\"><\/div>\n<p>If you want to try out GPT-3 today, you&rsquo;ll need to apply to be whitelisted by OpenAI. But the applications of this model seem endless &ndash; you could ostensibly use it to query a SQL database in plain English, automatically comment code, automatically&nbsp;generate&nbsp;code, write trendy article headlines, write viral Tweets, and a whole lot more.<\/p>\n<h2>But what&rsquo;s going on under the hood of this incredible model? Here&rsquo;s a (brief) look inside<\/h2>\n<p>GPT-3 is a neural-network-powered language model. A&nbsp;<a href=\"https:\/\/towardsdatascience.com\/language-modeling-c1cf7b983685\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">language model<\/a>&nbsp;is a model that predicts the likelihood of a sentence existing in the world. For example, a language model can label the sentence: &ldquo;I take my dog for a walk&rdquo; as more probable to exist (i.e. on the internet) than the sentence: &ldquo;I take my banana for a walk.&rdquo; This is true for sentences as well as phrases and, more generally, any sequence of characters.<\/p>\n<p>Like most language models, GPT-3 is elegantly trained on an unlabeled text dataset (in this case, the training data includes among others&nbsp;<a href=\"https:\/\/commoncrawl.org\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">Common Crawl<\/a>&nbsp;and Wikipedia). Words or phrases are randomly removed from the text, and the model must learn to fill them in using only the surrounding words as context. It&rsquo;s a simple training task that results in a powerful and generalizable model.<\/p>\n<p>The GPT-3 model architecture itself is a&nbsp;<a href=\"https:\/\/towardsdatascience.com\/https-medium-com-chaturangarajapakshe-text-classification-with-transformer-models-d370944b50ca\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">transformer-based<\/a>&nbsp;neural network. This architecture became popular around 2&ndash;3 years ago, and is the basis for the popular NLP model&nbsp;<a href=\"https:\/\/towardsdatascience.com\/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270\" target=\"_blank\" rel=\"nofollow noopener noreferrer\">BERT<\/a>&nbsp;and GPT-3&rsquo;s predecessor, GPT-2. From an architecture perspective, GPT-3 is not actually very novel!<\/p>\n<h2>What makes it so special and magical?<\/h2>\n<p>IT&rsquo;S REALLY BIG. I mean&nbsp;<em>really<\/em>&nbsp;big. With 175 billion parameters, it&rsquo;s the largest language model ever created (an order of magnitude larger than its nearest competitor!), and was trained on the largest dataset of any language model. This, it appears, is the main reason GPT-3 is so impressively &ldquo;smart&rdquo; and human-sounding.<\/p>\n<p>But here&rsquo;s the really magical part. As a result of its humongous size, GPT-3 can do what no other model can do (well): perform&nbsp;<em>specific<\/em>&nbsp;tasks without any special tuning. You can ask GPT-3 to be a translator, a programmer, a poet, or a famous author, and it can do it with its user (you) providing fewer than 10 training examples.&nbsp;<em>Damn<\/em>.<\/p>\n<p>This is what makes GPT-3 so exciting to machine learning practitioners. Other language models (like BERT) require an elaborate fine-tuning step where you gather&nbsp;<em>thousands<\/em>&nbsp;of examples of (say) French-English sentence pairs to teach it how to do translation. To adapt BERT to a specific task (like translation, summarization, spam detection, etc.), you have to go out and find a large training dataset (on the order of thousands or tens of thousands of examples), which can be cumbersome or sometimes impossible, depending on the task. With GPT-3, you don&rsquo;t need to do that fine-tuning step. This is the heart of it. This is what gets people excited about GPT-3: custom language tasks without training data.<\/p>\n<p>Today, GPT-3 is in private beta, but boy can I not wait to get my hands on it.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p class=\"tags\">\n<div><strong>See Campaign: <\/strong><a href=\"https:\/\/towardsdatascience.com\/language-modeling-c1cf7b983685\" target=\"_blank\">https:\/\/towardsdatascience.com\/language-modeling-c1cf7b983685<\/a><br \/><b>Contact Information:<\/b><br \/>Dale Markowitz<\/p>\n<p><b>Tags:<\/b><br \/><a href=\"\"><\/a>, <a href=\"https:\/\/icrowdnewswire.com\/category\/news-category\/wire\/\" rel=\"category tag\">Wire<\/a>, <a href=\"https:\/\/icrowdnewswire.com\/category\/global-regions\/united-states\/\" rel=\"category tag\">United States<\/a>, <a href=\"https:\/\/icrowdnewswire.com\/category\/language\/english\/\" rel=\"category tag\">English<\/a><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"\" alt=\"image\" width=\"400\" height=\"300\" class=\"cwdfimg\" \/><\/div>\n<div>\n<h3>Contact Information:<\/h3>\n<p>Dale Markowitz<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" width=\"233\" height=\"24\" src=\"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png\" alt=\"\">Peek under the hood of GPT-3 in under 3 minutes U So, you&rsquo;ve seen some amazing GPT-3 demos on Twitter (if not, where have you been?). This mega machine learning model, created by&nbsp;OpenAI, can write it&rsquo;s own op-eds, poems, articles, and even working code: If you want to try out GPT-3 today, you&rsquo;ll need to &hellip; <a href=\"https:\/\/icrowdnewswire.com\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/\">Continue reading <span>OpenAI&rsquo;s new GPT-3 language explained in under 3 minutes<\/span><\/a> <a href=\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/\" class=\"more-link\">Continue Reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":47,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,22,54],"tags":[],"class_list":["post-51946","post","type-post","status-publish","format-standard","hentry","category-english","category-united-states","category-wire"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.9 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>OpenAI\u2019s new GPT-3 language explained in under 3 minutes - Business<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"OpenAI\u2019s new GPT-3 language explained in under 3 minutes - Business\" \/>\n<meta property=\"og:description\" content=\"Peek under the hood of GPT-3 in under 3 minutes U So, you&rsquo;ve seen some amazing GPT-3 demos on Twitter (if not, where have you been?). This mega machine learning model, created by&nbsp;OpenAI, can write it&rsquo;s own op-eds, poems, articles, and even working code: If you want to try out GPT-3 today, you&rsquo;ll need to &hellip; Continue reading OpenAI&rsquo;s new GPT-3 language explained in under 3 minutes Continue Reading &rarr;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/\" \/>\n<meta property=\"og:site_name\" content=\"Business\" \/>\n<meta property=\"article:published_time\" content=\"2020-07-23T18:15:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png\" \/>\n<meta name=\"author\" content=\"Bilal\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Bilal\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/\",\"url\":\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/\",\"name\":\"OpenAI\u2019s new GPT-3 language explained in under 3 minutes - Business\",\"isPartOf\":{\"@id\":\"https:\/\/ipsnews.net\/business\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png\",\"datePublished\":\"2020-07-23T18:15:00+00:00\",\"author\":{\"@id\":\"https:\/\/ipsnews.net\/business\/#\/schema\/person\/70b05bacee6cf8a877350412fae25e20\"},\"breadcrumb\":{\"@id\":\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#primaryimage\",\"url\":\"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png\",\"contentUrl\":\"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ipsnews.net\/business\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"OpenAI\u2019s new GPT-3 language explained in under 3 minutes\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ipsnews.net\/business\/#website\",\"url\":\"https:\/\/ipsnews.net\/business\/\",\"name\":\"Business\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ipsnews.net\/business\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/ipsnews.net\/business\/#\/schema\/person\/70b05bacee6cf8a877350412fae25e20\",\"name\":\"Bilal\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/ipsnews.net\/business\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/01d05f3f41cc0f9ca88d2011a983bb3f2e83e3e92e3532188bf201df38d2aea8?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/01d05f3f41cc0f9ca88d2011a983bb3f2e83e3e92e3532188bf201df38d2aea8?s=96&d=mm&r=g\",\"caption\":\"Bilal\"},\"sameAs\":[\"https:\/\/icrowdnewswire.com\/fc\"],\"url\":\"https:\/\/ipsnews.net\/business\/author\/bilal\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"OpenAI\u2019s new GPT-3 language explained in under 3 minutes - Business","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/","og_locale":"en_US","og_type":"article","og_title":"OpenAI\u2019s new GPT-3 language explained in under 3 minutes - Business","og_description":"Peek under the hood of GPT-3 in under 3 minutes U So, you&rsquo;ve seen some amazing GPT-3 demos on Twitter (if not, where have you been?). This mega machine learning model, created by&nbsp;OpenAI, can write it&rsquo;s own op-eds, poems, articles, and even working code: If you want to try out GPT-3 today, you&rsquo;ll need to &hellip; Continue reading OpenAI&rsquo;s new GPT-3 language explained in under 3 minutes Continue Reading &rarr;","og_url":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/","og_site_name":"Business","article_published_time":"2020-07-23T18:15:00+00:00","og_image":[{"url":"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png","type":"","width":"","height":""}],"author":"Bilal","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Bilal","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/","url":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/","name":"OpenAI\u2019s new GPT-3 language explained in under 3 minutes - Business","isPartOf":{"@id":"https:\/\/ipsnews.net\/business\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#primaryimage"},"image":{"@id":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#primaryimage"},"thumbnailUrl":"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png","datePublished":"2020-07-23T18:15:00+00:00","author":{"@id":"https:\/\/ipsnews.net\/business\/#\/schema\/person\/70b05bacee6cf8a877350412fae25e20"},"breadcrumb":{"@id":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#primaryimage","url":"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png","contentUrl":"https:\/\/icrowdnewswire.com\/wp-content\/uploads\/2020\/06\/4001-logo.png"},{"@type":"BreadcrumbList","@id":"https:\/\/ipsnews.net\/business\/2020\/07\/23\/openais-new-gpt-3-language-explained-in-under-3-minutes\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ipsnews.net\/business\/"},{"@type":"ListItem","position":2,"name":"OpenAI\u2019s new GPT-3 language explained in under 3 minutes"}]},{"@type":"WebSite","@id":"https:\/\/ipsnews.net\/business\/#website","url":"https:\/\/ipsnews.net\/business\/","name":"Business","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ipsnews.net\/business\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/ipsnews.net\/business\/#\/schema\/person\/70b05bacee6cf8a877350412fae25e20","name":"Bilal","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ipsnews.net\/business\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/01d05f3f41cc0f9ca88d2011a983bb3f2e83e3e92e3532188bf201df38d2aea8?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/01d05f3f41cc0f9ca88d2011a983bb3f2e83e3e92e3532188bf201df38d2aea8?s=96&d=mm&r=g","caption":"Bilal"},"sameAs":["https:\/\/icrowdnewswire.com\/fc"],"url":"https:\/\/ipsnews.net\/business\/author\/bilal\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/posts\/51946","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/users\/47"}],"replies":[{"embeddable":true,"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/comments?post=51946"}],"version-history":[{"count":1,"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/posts\/51946\/revisions"}],"predecessor-version":[{"id":51947,"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/posts\/51946\/revisions\/51947"}],"wp:attachment":[{"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/media?parent=51946"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/categories?post=51946"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ipsnews.net\/business\/wp-json\/wp\/v2\/tags?post=51946"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}