{"id":9872,"date":"2021-05-28T14:26:56","date_gmt":"2021-05-28T18:26:56","guid":{"rendered":"https:\/\/glyphpress.com\/talk\/?p=9872"},"modified":"2021-06-04T14:02:30","modified_gmt":"2021-06-04T18:02:30","slug":"liberation-technology-3-the-making-of-a-cyborg","status":"publish","type":"post","link":"https:\/\/glyphpress.com\/talk\/2021\/liberation-technology-3-the-making-of-a-cyborg","title":{"rendered":"Liberation Technology 3: The Making of a Cyborg"},"content":{"rendered":"\n<p><em>You might want to start with <a href=\"https:\/\/www.patreon.com\/posts\/liberation-47985553\">the first Liberation Technology post <\/a>over on Punk Pedagogy!<\/em><\/p>\n\n\n\n<p>When Grace Hopper invented the first compiler \u2014 turning Roman characters typed on a Teletype machine into basic, digital computer instructions \u2014 it was a bold move toward making computation accessible to more humans than the tiny number of mathematicians who had both access to \u2014 and the talent for \u2014 writing the raw assembly code that described a computing process.<\/p>\n\n\n\n<!--more-->\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/external-content.duckduckgo.com\/iu\/?u=https%3A%2F%2F66.media.tumblr.com%2Fd686642e0b999ebda8d8c9a77d3aa7dc%2Ftumblr_p5ai6cpjP11suvpd0o2_640.jpg&amp;f=1&amp;nofb=1\" alt=\"Grace Hopper, American Computer Scientist Grace ...\"\/><figcaption>Grace Hopper did incredibly cool shit. We never discussed her or her work in a computer science class.<\/figcaption><\/figure>\n\n\n\n<p>She was using a recent, powerful human interface innovation, the Teletype, which was, itself, an adaptation of telegraph technology, but drawing from another recent information technology, the Linotype, used for laying out newspaper pages in lead type.<\/p>\n\n\n\n<p>The Linotype was an adaptation of the typewriter interface to the Gutenberg press, itself derived from woodblock presses that used single-purpose plates combined with design principles developed five centuries earlier in Chinese presses.<\/p>\n\n\n\n<p>In each case, the creators of a new medium found ways to integrate cutting-edge human interface with the creative powers of an earlier medium.<\/p>\n\n\n\n<p>And yet, in the year 2021, what we do consistently in programming is <em>emulate a teletype.<\/em> We use lines of code. In some languages, there\u2019s no particular meaning to a line break other than giving our eyes a rest. Some use color to distinguish types of commands or data, but few use anything other than <em>a one-dimensional list of Roman alphanumeric characters and punctuation<\/em> as a means of programming. To reduce the confusion, some IDEs color-coded text by type; rarely, they allow a part of the Unicode set of characters outside of those used by a Teletype of 1940. Capitalization conventions exist in many contradictory forms without an IDE even noticing, leading to errors, arguments, and public shaming, rather than a shared system of symbols.&nbsp;<\/p>\n\n\n\n<p>The front end of the Liberation Technology IDE has to be something that a new user can freely play with, implementing and debugging ideas, not chasing down syntax errors. When a creator makes a mistake, it must be an error of their design or their logic, not in their ability to accurately reproduce the rigid orthography required by a compiler, that requires their consideration.<\/p>\n\n\n\n<p>I\u2019ve seen a few ways of addressing such issues: HyperTalk\u2019s error-tolerant, flexibility; Scratch\u2019s (and its descendant, Blockly\u2019s) block model; Tina Quach\u2019s fault tolerant, speech-interpreting Codi; and MaxMSP (and other, usually musically oriented) patch model. I know what <em>I<\/em> want, but this is design, not art. It has to satisfy the needs of an entire audience, and in this case that intended audience is quite diverse.<\/p>\n\n\n\n<p>So let\u2019s talk about what a human-computer interface does first, and then let\u2019s talk about what these particular interfaces do, and then maybe what some hypothetical future interfaces might look like.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1038\" height=\"723\" src=\"https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-1038x723.jpeg\" alt=\"\" class=\"wp-image-9886\" srcset=\"https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-1038x723.jpeg 1038w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-300x209.jpeg 300w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-150x105.jpeg 150w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-768x535.jpeg 768w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-1536x1071.jpeg 1536w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-2048x1427.jpeg 2048w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-1200x836.jpeg 1200w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/1DE5AE98-E875-44CA-8153-F807FF4E0DDC-400x279.jpeg 400w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption>The human-computer interface feedback loop.<\/figcaption><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>A human-computer interface is a system that connects the nervous system of a user (in this case, a creator) to the capabilities of the machine in a feedback loop:<\/p>\n\n\n\n<p>We can start with the machine taking input: The machine takes in information from the user, uses that information according to its programming, then sends feedback information through the interface to the user\u2019s peripheral nervous system \u2014 through the eyes, if it uses a screen or lights; or through touch-sensitive nerves for haptic feedback; or through the ears with speakers.<\/p>\n\n\n\n<p>These inputs are connected to the user\u2019s central nervous and limbic systems and it uses those inputs to alter the user\u2019s perception.<\/p>\n\n\n\n<p>That perception informs them of how to send signals to their motor neurons, which then move their muscles to reorient parts of their body, from their eyes to fingers to sweat glands.<\/p>\n\n\n\n<p>The machine is able to perceive these changes, usually through the mechanical manipulation of potentiometers or switches, but sometimes also through galvanic sensors on the skin and, increasingly, cameras, microphones, and other, more ambiently-aware systems.<\/p>\n\n\n\n<p>The machine then takes this input and the feedback cycle repeats.<\/p>\n\n\n\n<p>Because this is a feedback system, you can \u201cstart\u201d at any point; it\u2019s an arbitrary design decision made according to the needs and assumptions of the design, but it is really important to know where to establish the initial conditions to that loop. For instance, where does the interaction loop with your phone begin? Probably when you first powered it on. It says \u201cHello.\u201d And then it\u2019s a process of years of your phone and you asking each other questions and giving each other answers. Just because it\u2019s in your pocket doesn\u2019t mean it\u2019s out of the loop; you give it motion and location data; you feel it in your pocket when someone texts you or when you\u2019re supposed to take out the garbage. You know how sometimes you put it in the wrong pocket or on the table, and then your usual pocket feels empty? That\u2019s because you\u2019ve left that feedback loop.<\/p>\n\n\n\n<p>In short, your phone is a <em>highly integrated<\/em> part of your nervous system. It\u2019s sometimes called a metacortex, even.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Programming: Telling the computer how to enter the feedback loop<\/h2>\n\n\n\n<p>For comparison, when you sit down to learn to program, what do you see? The Arduino IDE has a really interesting feature: it starts with two functions onscreen:<\/p>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<pre class=\"wp-block-preformatted\"><strong>void<\/strong> setup() {\n    \/\/ put your setup code here, to run once:\n}<\/pre>\n<\/div><\/div>\n\n\n\n<p>&#8230;and:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><strong>void<\/strong> loop() {\n    \/\/ put your main code here, to run repeatedly:\n}<\/pre>\n<\/div><\/div>\n\n\n\n<p>While C syntax is a real problem \u2014 a 50 year set of conventions that perfectly discourage most potential programmers \u2014 this particular choice of entry point is really smart. It tells a new programmer a couple of really important things:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Where do I put my code that I want to <strong>run as soon as I turn on<\/strong> my creation?<\/li><li>Where do I put my code that I want to <strong>run<\/strong> <strong>the whole time<\/strong> it\u2019s on?<\/li><li><strong>What a function looks like.<\/strong> That is, this code that shows when you select \u201cNew\u201d from the File menu implicitly demonstrates how <em>if(){}<\/em> works, and you\u2019re gonna need <em>if<\/em> almost immediately.<\/li><\/ol>\n\n\n\n<p>Because what\u2019s happening here is that the first move in that loop is being made by the design of the Arduino IDE. It\u2019s using its screen to show the user\u2019s eyes shapes that the user\u2019s brain assembles into words that tell the user\u2019s brain what structure to use next and how to use your hands to operate the keyboard and mouse to make things happen. Then, when you see things change onscreen, the loop continues.<\/p>\n\n\n\n<p>Once you\u2019ve written a few programs, you probably remember to delete the comments or insert your own. Eventually, you learn where global variables go (it\u2019s <strong>before<\/strong> <em>startup()<\/em>! It might be good to express <em>that<\/em> fact when you open your first blank file!) But I\u2019ve seen enormous, complex pieces of signal generation code that still include these comments. Hundreds of lines of code interpreting code from capacitive sensors, building waveforms, speaking MIDI, and error checking itself along the way. All, surrounded by the redundant, unnecessary \/\/<em>put your main code here, to run repeatedly:.&nbsp;<\/em> The initial comments clearly don\u2019t get in the way of experienced programmers.<\/p>\n\n\n\n<p>I think it would be good that startup file demonstrated some other, more optional basics, particularly <em>analogRead(), digitalRead(), analogWrite(), DigitalWrite()<\/em>, and example global variables and an <em>if()<\/em> statement. But at that point, we\u2019re adding yet another patch to a language design that\u2019s almost twice as old as the average human. <strong>We\u2019re not going to solve this problem with additional duct tape.<\/strong><\/p>\n\n\n\n<p>Let\u2019s look at some other ways that people have interacted with this programming interaction feedback loop.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">HyperTalk: zooming from object to script to C<\/h2>\n\n\n\n<p>HyperTalk was the programming language behind HyperCard \u2014 the development environment included with every Mac from the late 80s through mid-90s. It was the first \u201cscripting language\u201d and featured the little gloved Mickey Mouse hand you now know from Web browsers.<\/p>\n\n\n\n<p>When your opened up HyperCard, you probably did it through a \u201cStack\u201d, an executable database containing both code and data. You were probably opening one of the included, useful stacks, like the Rolodex that was where most users experienced all they ever would.<\/p>\n\n\n\n<p>But eventually, many users would realized that they needed an additional field. Perhaps they needed to add an additional phone number field for the fax machine.<\/p>\n\n\n\n<p>So they would add phone number field. They\u2019d draw it as a rectangle into the background card, and then it would work.<\/p>\n\n\n\n<p>And maybe you\u2019d double click a button while you were editing the fields, and see that the \u201cNew Card\u201d button had a little script in it that said something like,&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">on mouseUp\n    new card after this card\nend mouseUp<\/pre>\n\n\n\n<p>And it would start to dawn on you that you could tell it to do stuff. \u201cWhat if I want to make sure I\u2019m not making a duplicate card?\u201d you might think. So then you could look at how to do logic when you entered text into a field.<\/p>\n\n\n\n<p>In many cases, it wasn\u2019t long before that Rolodex was talking with your calendar stack so you could say who your phone call was with. And the stack was dialing the phone for you.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1038\" height=\"645\" src=\"https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-1038x645.png\" alt=\"\" class=\"wp-image-9902\" srcset=\"https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-1038x645.png 1038w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-300x186.png 300w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-150x93.png 150w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-768x477.png 768w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-1200x746.png 1200w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-400x249.png 400w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-97x60.png 97w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image-145x90.png 145w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2021\/05\/image.png 1284w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption>The structure of HyperCard. Code in any deeper layer could be accessed by higher layers, so you&#8217;d put functions into <strong>Home<\/strong> that you&#8217;d need for all your stacks, the code you needed for your particular stack only in that stack. Stacks contained backgrounds could contain objects, including their code, for all the objects on the cards that use that background, and then each card contained objects that could hold their own code. So when you coded a button on a card, you might just say <em>on mouseUp sort end mouseUp<\/em>, knowing that the<em> sort<\/em> function was something you&#8217;d coded fully in a deeper level.<\/figcaption><\/figure>\n\n\n\n<p>The interaction loop started when you started using built-in software that allowed you \u2014 invited you \u2014 to open up the hood and see how each button, field, card, stack, and sound effect worked. You\u2019d click a button that  did something clever, and you\u2019d open it, and the answer was <em>right there<\/em>. It just <em>showed<\/em> how the programmer did it. And then you knew. Or you didn\u2019t understand it, but you\u2019d copy and paste it and change a thing or two, and then it worked. Or it didn\u2019t and you\u2019d have to debug, learning the whole time. Maybe you\u2019d use a book, or maybe you\u2019d learn something from your companions on a HyperCard listserv.<\/p>\n\n\n\n<p>Eventually, you\u2019d hit one of HyperCard\u2019s pretty serious limits and start hunting for an external plugin \u2014 an XCMD, an eXternal CoMmanD \u2014 to extend it, written in C. You\u2019d install the external into your stack, then go about making your thing.<\/p>\n\n\n\n<p>And sometimes, just sometimes you\u2019d learn how to write that XCMD yourself, applying C to a high-level language, extending it for your own use and that of others. There was even a stack called<em> CompileIt!<\/em> that helped you write XCMD that linked to the Macintosh OS&#8217;s APIs, allowing you OS-level programming.<\/p>\n\n\n\n<p>As HyperCard exists, one\u2019s objects (e.g. buttons, fields) can call functions from a card, which can call functions from the background it shares with other cards, which can call functions from the stack shared by all the cards and backgrounds, which can call external functions installed into the stack by the stack author. Liberation Technology builds on this model of \u201czooming in\u201d on objects to see a lower level of programming abstraction inside.<\/p>\n\n\n\n<p>HyperCard never intended to dig all the way down to low-level programming, but it provided the framework for external, compiled executable code that could be called from within. Liberation Technology intends to fully connect the creative processes from high level authoring to scripting to high performance coding.<\/p>\n\n\n\n<p>However, HyperTalk\u2019s predecessor, SmallTalk, did. Its intended function was to <em>be<\/em> the operating system. You could always open something up and change how it work. <em>Anything<\/em>. Sadly, while it\u2019s a beloved programming environment by its enthusiasts, the OS built around it was trapped inside the experimental XEROX PARC project and never became an affordable computer. There was a movement inside Apple when to make HyperCard the native language of the Macintosh, but its power to <em>show the user how it works<\/em> interfered with the then-nascent idea of software intellectual property. I think it\u2019s questionable as to whether the SmallTalk model can exist alongside patented software, or, indeed consumerism.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Scratch: doing away with syntax errors<\/h2>\n\n\n\n<p>Scratch uses a \u201cblock\u201d model of programming to model indented code, where the curly braces in <em>if(){}<\/em> are part of a virtual object and the parentheses are fields that are the particular shape of a logical condition, which only fits if it\u2019s syntactically appropriate, like a triangle block into a triangle hole. You can\u2019t delete the second curly brace or indent it incorrectly because it\u2019s just the other side of the <em>if<\/em> object. Other objects \u2014 other logical structures \u2014 are held within it or encapsulate it by wrapping around it, or they operate independently in another part of the code altogether.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/external-content.duckduckgo.com\/iu\/?u=https%3A%2F%2F1.bp.blogspot.com%2F-g_tGfwB_Xj8%2FXj6wFuPfagI%2FAAAAAAAAP1c%2FTOxyOPbqWG4OmepEMH3jri3hFAGlfv0kACLcBGAsYHQ%2Fs1600%2Fscratch-screen.png&amp;f=1&amp;nofb=1\" alt=\"Introduction to Scratch Programming for ICSE Class 5\"\/><figcaption>The structure of a blank Scratch program when you first open it up.<\/figcaption><\/figure>\n\n\n\n<p>That code typically works inside the <em>stage <\/em>(the environment) containing an <em>actor<\/em> (an object containing code) that probably has a <em>sprite<\/em> (a representation that you can see on the stage) that has its own code. Actors are intended to act as agents, running their own code and interacting by touching objects or colors, by calculating the direction of something else to their own center, and generally acting like characters and environmental features in a video game.<\/p>\n\n\n\n<p>What this does at its best is assure that any code you write (build?) will run. It will do <em>something<\/em>. And that means that you\u2019re debugging your ideas, your algorithms, your processes, rather than hunting for syntax errors.<\/p>\n\n\n\n<p>Additionally, you can see the interactions and where they go wrong. You will often know that the problem is within a certain actor because you know something goes wrong when it touches another actor. So you know to look in those two actors first to figure out if the error is within their code.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/external-content.duckduckgo.com\/iu\/?u=http%3A%2F%2Fwww.snapfiles.com%2Fscreenfiles%2Fscratch.png&amp;f=1&amp;nofb=1\" alt=\"Scratch screenshot and download at SnapFiles.com\"\/><figcaption>A Scratch program<\/figcaption><\/figure>\n\n\n\n<p>A really important aspect of Scratch is that, on the Scratch website, you can look inside any Scratch program you find to see how the creator did whatever it is the program does. You can find solutions to problems by copying their code, find new inspiration and understanding, and improve on other people\u2019s work.<\/p>\n\n\n\n<p>Scratch, however, assumes that the best way to interact with a computer is by writing concatenated text, and that you just need help. It works to reduce typos by typing the text for you as you select from a palette. It\u2019s a great way to get started, but it becomes cumbersome and illegible when a program starts to get complex.<\/p>\n\n\n\n<p>It uses the 2D nature of a computer screen to allow you to organize chunks of code as you see fit, but the arrangement is stylistic, not functional. It makes no difference how close or in what direction pieces of code are; or, indeed if those chunks of code even work. One will often find non-running examples alongside working code, as well as broken snippets that have been removed from the running code during the debugging process and have never been tidied up. And it often looks just like the working, running code beside it.<\/p>\n\n\n\n<p>But the biggest issue I&#8217;ve noticed when teenagers with Scratch is that it announces, asserts, demands that <em>Scratch is For Kids<\/em>. That gives it both an entry into the creator-computer feedback loop, but also insists on an exit point.<\/p>\n\n\n\n<p>When a small child of, say, 8 comes across Scratch, it feels much like LEGO (who have a partnership with Scratch), where you can assemble things out of raw parts within creative constraints. It works! Hooray! They start learning programming concepts right away! Kids get into systems <em>deep<\/em>, and they\u2019ll dig into Scratch like it\u2019s Minecraft.<\/p>\n\n\n\n<p>But if there\u2019s one thing kids hate, it\u2019s getting stuck using things that are for kids. They&#8217;ll hit its limits, and kids always want to use real tools. If you want a kid to learn how to use a tool, you give one that\u2019s sized for their hand, not a dull knife and a soft hammer to keep them safe. You give them small tools and teach them how they work, and you teach them how to recover from cutting themselves or hitting themselves on the thumb.&nbsp;<\/p>\n\n\n\n<p>Scratch says firmly that, if you want to make a self-running program \u2014 one that runs on your phone or on a robot of your own design \u2014 that it is time to <em>stop using Scratch<\/em>.<\/p>\n\n\n\n<p>To be sure, Scratch\u2019s basic engine is being used in projects like Blockly that convert Scratch constructions to other forms of code, which links between the two. But those projects have been begun and been abandoned often because <em>the developers of those environments are not their users.<\/em> Eric Raymond points out in The Cathedral and The Bazaar that developers improving their own tools, and the tools being built by people who use them, is critical to the long term health of an Open Source project. <\/p>\n\n\n\n<p>Furthermore, Scratch and its descendant environments make the assumption that the fundamental form of code is in a one-dimensional list of characters. Even if we accept that this is a reasonable way for computer code to look, there\u2019s no reason for a programming front end to follow this convention. There\u2019s nothing inherent about a flow of logic that requires the particular form. It\u2019s a matter of conceptual backward compatibility, not a matter of wisely designed interaction.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Max, Nord, and Axoloti: patch the planet<\/h2>\n\n\n\n<p>In the music world, a popular metaphor for signal flow is the \u201cpatch\u201d \u2014 a representation of wires, like one that runs from a guitar to a pedal to an amplifier.<\/p>\n\n\n\n<p>Wires are a pretty solid metaphor: waves of changing voltage come out of one system \u2014 the pickups on an electric guitar \u2014 then flows into another \u2014 the pedal. The pedal changes the shape of the waves, generating a signal from the incoming sound and outputting it to a jack just like the input jack. Then, the cable runs from the pedal to the amplifier, which gives the signal enough power to move a speaker cone to generate sound.<\/p>\n\n\n\n<p>What\u2019s particularly neat about this metaphor is that the pedal can do anything a pedal designer is interested in. You can plug a guitar into it, or a microphone, or another pedal. Some pedals do much more than alter the shape of the waveforms. Some play a controllable echo. Others synchronize other instruments to the same rhythm or alter lighting according to the sound being produced.<\/p>\n\n\n\n<p>This metaphor is taken to its extreme in modular sound synthesis, where there needs to be no initial input of sound. Signals are generated, processed, and reproduced electronically. By plugging the output of one module into another, we can generate a vast, extremely complex signal system.<\/p>\n\n\n\n<p>Such systems have been reproduced in digital form since the 1990s, but Max and the Nord Modular began treating the patches as a computer program, rather than a computer simulation of an analog signal system. Those systems are fairly prohibitive in cost, but have inspired many free and low-cost alternatives, like Pure Data and Axoloti.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/external-content.duckduckgo.com\/iu\/?u=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FIlias_Bergstrom%2Fpublication%2F271190788%2Ffigure%2Ffig1%2FAS%3A279743212605440%401443707387093%2FMax-MSP-patch-showing-data-processing-and-data-generating-boxes-connected-to-each-other.png&amp;f=1&amp;nofb=1\" alt=\"Max\/MSP patch, showing data-processing and data-generating ...\"\/><figcaption>A Max\/MSP patch<\/figcaption><\/figure><\/div>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/external-content.duckduckgo.com\/iu\/?u=https%3A%2F%2Fwww.sequencer.de%2Fpix%2Fclavia%2Ffiltertrix.gif&amp;f=1&amp;nofb=1\" alt=\"clavia nord modular patch tricks\"\/><figcaption>A Nord Modular virtual patch<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/external-content.duckduckgo.com\/iu\/?u=http%3A%2F%2Fcommunity.axoloti.com%2Fuploads%2Fdefault%2Foriginal%2F2X%2Ff%2Ff8b5888404ed2affa468360df3d2bad57a454cc5.JPG&amp;f=1&amp;nofb=1\" alt=\"Basic polyphonic synth - The Hera - Patching - Axoloti ...\"\/><figcaption>An Axoloti patch<\/figcaption><\/figure>\n\n\n\n<p>To program an Axoloti synthesizer development board, the creator arranges modules and connects them by \u201cwires\u201d, color coded for their sample rate. To take input from an external MIDI, one runs a wire from a MIDI module\u2019s <em>note output <\/em>to the <em>pitch<\/em> input on an oscillator, and the <em>gate<\/em> output to an amplifier module that also takes in the oscillator output.<\/p>\n\n\n\n<p>You can see, in that paragraph, why it is better to draw the idea than to write it out in a list of characters.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Codi: a Coding Agent<\/h2>\n\n\n\n<p>Tina Quach introduced a program called Codi in her paper, <strong><a href=\"https:\/\/www.media.mit.edu\/publications\/agent-based-programming-interfaces-for-children-supporting-blind-children-in-creative-computing-through-conversation\/\">Agent-based Programming Interfaces for Children: <\/a><\/strong><em><a href=\"https:\/\/www.media.mit.edu\/publications\/agent-based-programming-interfaces-for-children-supporting-blind-children-in-creative-computing-through-conversation\/\">Supporting Blind Children in Creative Computing through Conversation<\/a>.<\/em> Codi is a speech-based code editor that listens to speech instructions and gives back speech feedback for existing coding structures. Because parentheses, brackets, carats, and braces are a particular typographical convention that does not follow any particular speech pattern, Quach had to invent new descriptive structures for programming.<\/p>\n\n\n\n<p><em>When the space key is pressed, repeat the following three times. Play the hi na tabla sound. Wait 0.25 seconds. Play the hi tun tabla sound. Wait 0.25 seconds. That\u2019s it.<\/em><\/p>\n\n\n\n<p>The \u201cThat\u2019s it.\u201d Command tells Codi that this is the end of a loop. \u201cRepeat the following three times\u201d tells the loop how many times to do it, of course, but the most interesting thing to me is that it starts with \u201cWhen the space key is pressed.\u201d<\/p>\n\n\n\n<p>Codi starts a behavior at an event. In this case, it\u2019s an event that comes from an input device, but there\u2019s no reason it couldn\u2019t come from an internal process.<\/p>\n\n\n\n<p>In this regard, it\u2019s a lot like a patcher environment, where the model is that&nbsp; signal flows through the program, starting with an event that precipitates other events.<\/p>\n\n\n\n<p>Contrast that with this pseudocode:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><em>keyboard.listen;<\/em>\n    <em>if (keyboard.listen(keyPressed) = \" \"){<\/em>\n    <em>for (i=0, i=3, <em>i=i+1<\/em>){<\/em>\n         <em>sound.play (\u201chi na tabla\u201d);<\/em>\n         <em>delay (250);<\/em>\n         <em>sound.play (\u201chi tun tabla\u201d)<\/em>\n         <em>delay (250);<\/em>\n    <em>{<\/em>\n<em>}<\/em><\/pre>\n\n\n\n<p>In this C-like pseudocode, we\u2019re looking around to see what is true. We\u2019re not starting with an event; the program is ignorant of what we care about, so we have to tell it to look at what keys are being pressed by polling all the keys, then implicitly ignore all the keys that aren\u2019t the one we\u2019re looking at, then tell it how to count to three.<\/p>\n\n\n\n<p>(Did you catch the syntax error in that pseudocode?)<\/p>\n\n\n\n<p>I think event triggering \u2014&nbsp;perhaps constructing processes out of, well, <em>processes<\/em> rather than objects \u2014 might really empower new and casual creators, drawing on our common perception of events in the rest of the world.<\/p>\n\n\n\n<p>We don\u2019t check every object in the room to see which one\u2019s we\u2019re interacting with. We interact with the ones that stimulate our sensory system: the ones we see, feel, hear, taste, and smell.<\/p>\n\n\n\n<p>OK, yes, you&#8217;re right: there aren\u2019t a lot of human-computer interfaces that use taste or smell.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Treating Programming as a Regular Human-Computer Interface<\/h2>\n\n\n\n<p>By using the principles and techniques of UX, from <a href=\"https:\/\/uxdesign.cc\/the-curb-cut-effect-universal-design-b4e3d7da73f5\">Universal Design<\/a> to iterative user testing, I think we can change our relationship as a society to our programmed environment \u2014 one that is becoming as pervasive as sidewalk curbs. None of us have been able to benefit yet from a programming interface design parallel to the curb cut.<\/p>\n\n\n\n<p>Grace Hopper famously said, \u201cIt\u2019s much easier for most people to write an English statement than to use symbols.\u201d That priority has only become more critical as programming has become the provice of a scribe class, rather than becoming a quotidian activity as promised at the dawn of the Personal Computer revolution. The result has been that we purchase computers that are programmed to do what the programmers\u2019 employer wants them to do within the feedback loop, not what the user wants.<\/p>\n\n\n\n<p>There are real questions to be addressed in the Liberation Technology interface design process. How to we use Universal Design principles to help the most experienced creators \u2014&nbsp;dedicated and professional programmers \u2014 benefit by a design centering on the experience of those who have the fewest advantages \u2014 kids, the dyslexic, the neurodivergent, the blind? I feel pretty strongly that these questions are the doorways to a new kind of sociotechnological experience.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1080\" height=\"209\" src=\"https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2016\/08\/image.png\" alt=\"\" class=\"wp-image-3360\" srcset=\"https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2016\/08\/image.png 1080w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2016\/08\/image-300x58.png 300w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2016\/08\/image-768x149.png 768w, https:\/\/glyphpress.com\/talk\/wp-content\/uploads\/2016\/08\/image-1024x198.png 1024w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><figcaption class=\"wp-element-caption\">Modular systems are a function of industrial society. But do people of The Fifth World still know how to agree to standards? With their acute interest in efficiency, I think they might have carried that lesson forward!<\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>You might want to start with the first Liberation Technology post over on Punk Pedagogy! When Grace Hopper invented the first compiler \u2014 turning Roman characters typed on a Teletype machine into basic, digital computer instructions \u2014 it was a bold move toward making computation accessible to more humans than the tiny number of mathematicians &hellip; <a href=\"https:\/\/glyphpress.com\/talk\/2021\/liberation-technology-3-the-making-of-a-cyborg\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Liberation Technology 3: The Making of a Cyborg&#8221;<\/span><\/a><\/p>\n","protected":false},"author":35,"featured_media":9886,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-9872","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-set-of-all-sets-that-do-not-include-themselves"],"_links":{"self":[{"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/posts\/9872","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/users\/35"}],"replies":[{"embeddable":true,"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/comments?post=9872"}],"version-history":[{"count":6,"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/posts\/9872\/revisions"}],"predecessor-version":[{"id":9936,"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/posts\/9872\/revisions\/9936"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/media\/9886"}],"wp:attachment":[{"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/media?parent=9872"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/categories?post=9872"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/glyphpress.com\/talk\/wp-json\/wp\/v2\/tags?post=9872"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}