Joel Brandt
joel.brandt@adobe.com · https://joelbrandt.com/
Education
2010
Stanford University · Ph.D. in Computer Science
Thesis title: Example-Centric Programming: Integrating Web Search into the Development Process
Advisor: Scott R. Klemmer, Readers: Mira Dontcheva, John Ousterhout, Terry Winograd
2005
Washington University in St. Louis · M.S. in Computer Science
Thesis title: What A Mesh: Dependent Data Types for Correct Mesh Manipulation Algorithms
Readers: Aaron Stump, Sally Goldman, Cindy Grimm
2005
Washington University in St. Louis · B.S., dual majors in Computer Science and Applied Mathematics
Magna Cum Laude, GPA 4.0/4.0
Employment
2019 –
Adobe Research · Los Angeles, CA
Principal Scientist
Invent new technologies and interactions that revolutionize the way people create.
2017 – 2019
Snap, Inc. · Los Angeles, CA
Sr. Manager, Research Engineering
Lead advanced engineering team focused on creating new technology and products.
2010 – 2017
Adobe Research · San Francisco, CA
Director, Research Engineering
Lead advanced engineering team focused on creating new technology and products.
2003 – 2004
Mass Sensors, Inc. · St. Louis, Missouri
Software Engineer
Develop software and firmware for chemical analysis equipment.
Internships
2008 – 2010
Adobe Creative Technologies Lab · San Francisco, CA
Research Intern
Understand and improve how programmers use the Web.
2006
Google · Mountain View, CA
Intern
Develop new ranking algorithms for personalized Web search, focusing on user location.
2005
Metris, Inc. (now Nikon, Inc.) · Dulles, VA
Application Software Engineer
Develop software for metrology solutions.
Publications
Full Papers
2020
Temporal Segmentation of Creative Live Streams [doi]
C. Ailie Fraser, Joy O. Kim, Hijung Valentina Shin, Joel Brandt, Mira Dontcheva
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Supporting Visual Artists in Programming through Direct Inspection and Control of Program Execution [doi]
Jingyi Li, Joel Brandt, Radomír Mech, Maneesh Agrawala, Jennifer Jacobs
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2019
Implementing Multi-Touch Gestures with Touch Groups and Cross Events (Best Paper Honorable Mention) [local]
Steve Oney, Rebecca Krosnick, Joel Brandt, Brad Myers
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2018
Extending Manual Drawing Practices with Artist-Centric Programming Tools (Best Paper) [local]
Jennifer Jacobs, Joel Brandt, Radomír Měch, Mitchel Resnick
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Rewire: Interface Design Assistance from Examples [local]
Amanda Swearngin, Mira Dontcheva, Wilmot Li, Joel Brandt, Morgan Dixon, Andrew J. Ko
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2017
Supporting Expressive Procedural Art Creation through Direct Manipulation (Best Paper) [local] [video]
Jennifer Jacobs, Sumit Gogia, Radomír Měch, Joel Brandt
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2016
Using Runtime Traces to Improve Documentation and Unit Test Authoring for Dynamic Languages [local] [video]
Jan-Peter Krämer, Joel Brandt, Jan Borchers
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2015
Selective Undo Support for Painting Applications [local] [video]
Brad A. Myers, Ashley Lai, Tam Minh Le, YoungSeok Yoon, Andrew Faulring, Joel Brandt
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2014
InterState: A Language and Environment for Expressing Interface Behavior [local] [video]
Stephen Oney, Joel Brandt, Brad Myers
UIST: ACM Symposium on User Interface Software and Technology
Addressing Misconceptions About Code with Always-On Programming Visualizations [local] [video]
Tom Lieber, Joel Brandt, Robert C. Miller
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Combining Crowdsourcing and Learning to Improve Engagement and Performance [local]
Mira Dontcheva, Robert Morris, Joel Brandt, Elizabeth M. Gerber
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Emergent, Crowd-scale Programming Practice in the IDE [local]
Ethan Fast, Daniel Steffee, Lucy Wang, Joel Brandt, Michael S. Bernstein
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2012
ConstraintJS: Programming Interactive Behaviors for the Web by Integrating Constraints and States [local] [video]
Stephen Oney, Joel Brandt, Brad Myers
UIST: ACM Symposium on User Interface Software and Technology
SnipMatch: Using Source Code Context to Enhance Snippet Retrieval and Parameterization [local] [video]
Doug Wightman, Zi Ye, Joel Brandt, Roel Vertegaal
UIST: ACM Symposium on User Interface Software and Technology
Codelets: Linking Interactive Documentation and Example Code in the Editor [local] [doi] [video]
Stephen Oney, Joel Brandt
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Analytic Methods for Optimizing Realtime Crowdsourcing [local] [arXiv]
Michael S. Bernstein, Joel Brandt, David R. Karger, Robert C. Miller
CI: Collective Intelligence
2011
Crowds in Two Seconds: Enabling Realtime Crowd-Powered Interfaces [local] [doi] [video]
Michael S. Bernstein, Joel Brandt, Robert C. Miller, and David R. Karger
UIST: ACM Symposium on User Interface Software and Technology
2010
What Would Other Programmers Do? Suggesting Solutions to Error Messages (Best Paper Honorable Mention) [local] [doi] [video]
Bjoern Hartmann, Daniel MacDougall, Joel Brandt, Scott R. Klemmer
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Example-Centric Programming: Integrating Web Search into the Development Environment (Best Paper Honorable Mention) [local] [doi] [video]
Joel Brandt, Mira Dontcheva, Marcos Weskamp, Scott R. Klemmer
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2009
Two Studies of Opportunistic Programming: Interleaving Web Foraging, Learning, and Writing Code (Best Paper Honorable Mention) [local] [doi]
Joel Brandt, Philip Guo, Joel Lewenstein, Mira Dontcheva, Scott R. Klemmer
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Opportunistic Programming: Writing Code to Prototype, Ideate, and Discover [local] [doi]
Joel Brandt, Philip Guo, Joel Lewenstein, Mira Dontcheva, Scott R. Klemmer
IEEE Software, 26 (5), 18-24.
Short Papers
2013
Creating Small Products at a Big Company: Adobe’s “Pipeline” Innovation Process [local]
Rob J. Adams, Bradee Evans, Joel Brandt
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2012
A Print Magazine on Any Screen: The Wired App Story [local] [doi]
Jeremy Clark, Joel Brandt
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
Workshops, Posters, & Demonstrations
2018
Dynamic Brushes: Extending Manual Drawing Practices with Artist-Centric Programming Tools [doi]
Jennifer Jacobs, Joel R. Brandt, Radomír Měch, Mitchel Resnick
CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2016
Digital Craftsmanship: HCI Takes on Technology as an Expressive Medium [local]
Jennifer Jacobs, David Mellis, Amit Zoran, Cesar Torres, Joel Brandt, Joshua Tanenbaum
Proposal for accepted workshop at DIS: ACM SIGCHI Conference on Designing Interactive Systems
An Empirical Study of Programming Paradigms for Animation [local]
Jan-Peter Kraämer, Michael Hennings, Joel Brandt, Jan Borchers
CHASE 2016: Cooperative and Human Aspects of Software Engineering (Workshop at ICSE 2016)
2013
Euclase: A Live Development Environment with Constraints and FSMs [local]
Stephen Oney, Brad Myers, Joel Brandt
LIVE 2013: First International Workshop on Live Programming (Workshop at ICSE 2013)
Creativity Support in Authoring and Backtracking [local]
Brad A. Myers, Stephen Oney, YoungSeok Yoon, Joel Brandt
ECSE 2013: Evaluation Methods for Creativity Support Environments (Workshop at CHI 2013)
2012
Photoshop with Friends: A Synchronous Learning Community for Graphic Design [local] [doi]
Juho Kim, Benjamin Malley, Joel Brandt, Mira Dontcheva, Diana Joseph, Krzysztof Z. Gajos, Robert C. Miller
Demonstration at CSCW: ACM Conference on Computer Supported Cooperative Work
2009
Remixing The Web: Enhancing Tailoring Using Programmable Proxies [local]
Joel Brandt, Leslie Wu, Scott R. Klemmer
CHI Workshop on End User Programming for the Web
2008
Rehearse: Coding Interactively while Prototyping [local] [video]
William Choi, Joel Brandt, Scott R. Klemmer
Poster at UIST: ACM Symposium on User Interface Software and Technology
Opportunistic Programming: How Rapid Ideation and Prototyping Occur in Practice [local]
Joel Brandt, Philip J. Guo, Joel Lewenstein, Scott R. Klemmer
Fourth Workshop on End-User Software Engineering at ICSE: International Conference on Software Engineering
2007
txt 4 l8r: lowering the burden for diary studies under mobile conditions [local] [doi]
Joel Brandt, Noah Weiss, Scott R. Klemmer
Work-in-Progress at CHI: ACM SIGCHI Conference on Human Factors in Computing Systems
2006
Lash-Ups: A Toolkit for Location-Aware Mash-Ups [local] [poster] [video]
Joel Brandt, Scott R. Klemmer
Poster at UIST: ACM Symposium on User Interface Software and Technology
Interactive Gigapixel Prints: Large, Paper-Based Interfaces for Visual Context and Collaboration [local] [video]
Ron Yeh, Joel Brandt, Jonas Boli, and Scott R Klemmer
Video demonstration at UbiComp: International Conference on Ubiquitous Computing
Theses
2010
Example-Centric Programming: Integrating Web Search into the Development Process [local]
PhD Thesis, Stanford University
Advisor: Scott R. Klemmer, Readers: Mira Dontcheva, John Ousterhout, Terry Winograd
2005
What A Mesh: Dependent Data Types for Correct Mesh Manipulation Algorithms [local]
Masters Thesis, Washington University in St. Louis
Readers: Aaron Stump, Cindy Grimm, Sally Goldman
Book Chapters
2010
How the Web Helps People Turn Ideas Into Code
Joel Brandt, Philip Guo, Joel Lewenstein, Mira Dontcheva, Scott R. Klemmer
In No Code Required: Giving Users Tools to Transform the Web, edited by Cypher, A., Dontcheva, M., Lau, T., Nichols, J., Kaufmann, M
Patents Issued
2023
Video Search Segmentation
Hijung Shin, Cristin Ailidh Fraser, Aseem Agarwala, Lubomira Dontcheva, Joel Richard Brandt, Jovan Popovic
US Patent No. 11,810,358 · Issued November 7, 2023
Interacting With Hierarchical Clusters Of Video Segments Using A Metadata Search
Seth Walker, Joy Oakyung Kim, Morgan Nicole Evans, Najika Skyler Halsema Yoo, Aseem Agarwala, Joel Richard Brandt, Jovan Popovic, Lubomira Dontcheva, Dingzeyu Li, Hijung Shin, Xue Bai
US Patent No. 11,762,903 · Issued September 19, 2023
Selecting And Performing Operations On Hierarchical Clusters Of Video Segments
Seth Walker, Joy Oakyung Kim, Aseem Agarwala, Joel Richard Brandt, Jovan Popovic, Lubomira Dontcheva, Dingzeyu Li, Hijung Shin, Xue Bai
US Patent No. 11,631,434 · Issued April 18, 2023
Interacting With Hierarchical Clusters Of Video Segments Using A Video Timeline
Seth Walker, Joy Oakyung Kim, Aseem Agarwala, Joel Richard Brandt, Jovan Popovic, Lubomira Dontcheva, Dingzeyu Li, Hijung Shin, Xue Bai
US Patent No. 11,630,562 · Issued April 18, 2023
2022
Video Segmentation based on Detected Video Features using a Graphical Model
Hijung Shin, Cristin Ailidh Fraser, Aseem Agarwala, Lubomira Dontcheva, Joel Richard Brandt, Jovan Popovic
US Patent No. 11,455,731 · Issued September 27, 2022
Segmentation and Hierarchical Clustering of Video
Hijung Shin, Xue Bai, Aseem Agarwala, Joel R. Brandt, Jovan Popovic, Lubomira Dontcheva, Dingzeyu Li, Joy Oakyung Kim, Seth Walker
US Patent No. 11,450,112 · Issued September 20, 2022
2021
(Continuation) Inter-Context Coordination to Facilitate Synchronized Presentation of Image Content
Ian Wehrman, John Fitzgerald, Joel Brandt, Jesper Bache, David Tristram, Barkin Aygun
US Patent No. 10,908,764 · Issued February 2, 2021
Video Reformatting System
Joel Brandt, Giles Goodwin, Jared Iwata, Ian Wehrman
US Patent No. 10,887,542 · Issued January 5, 2021
2019
(Continuation) Undo, Redo, and State Switching Using Content-Addressable Data Structures
David P. Simons, James Acquavella, Gregory Evans, Joel Brandt
US Patent No. 10,402,481 · Issued September 3, 2019
User Interface Creation from Screenshots
Morgan Dixon, Lubomira Dontcheva, Joel Brandt, Amanda Swearngin
US Patent No. 10,360,473 · Issued July 23, 2019
2018
Inter-Context Coordination to Facilitate Synchronized Presentation of Image Content
Ian Wehrman, John Fitzgerald, Joel Brandt, Jesper Bache, David Tristram, Barkin Aygun
US Patent No. 10,073,583 · Issued September 11, 2018
2017
Undo, Redo, and State Switching Using Content-Addressable Data Structures
David P. Simons, James Acquavella, Gregory Evans, Joel Brandt
US Patent No. 9,842,094 · Issued December 12, 2017
2016
Methods and Apparatus for Code Segment Handling
Joel Brandt, Stephen Oney
US Patent No. 9,495,134 · Issued November 15, 2016
Patents Pending
2022
(Continuation) Interacting With Hierarchical Clusters Of Video Segments Using A Video Timeline
Filed October 19, 2022
Video Segment Selection And Editing Using Transcript Interactions
Filed internationally in Australia, China, and Germany
Filed October 17, 2022
Transcript Question Search For Text-Based Video Editing
Filed October 17, 2022
Display Screen Or Portion Thereof With Graphical User Interface
Filed October 17, 2022
Speaker Thumbnail Selection And Speaker Visualization In Diarized Transcripts For Text-Based Video
Filed October 17, 2022
Visual And Text Search Interface For Text-Based Video Editing
Filed October 17, 2022
Annotated Transcript Text And Transcript Thumbnail Bars For Text-Based Video Editing
Filed October 17, 2022
Transcript Paragraph Segmentation And Visualization Of Transcript Paragraphs
Filed October 17, 2022
(Continuation) Modifying A Default Video Segmentation
Filed June 8, 2022
(Continuation) Hierarchical Segmentation Based On Voice-Activity Detection
Filed June 2, 2022
(Continuation) Hierarchical Segmentation Based Software Tool Usage In A Video
Filed June 2, 2022
(Continuation) Hierarchical Segmentation Of Screen Captured, Screencasted, Or Streamed Video
Filed June 2, 2022
Interacting With Semantic Video Segments Through Interactive Tiles
Filed internationally in Australia, China, Germany, and United Kingdon
Filed May 26, 2022
2021
Snap Point Video Segmentation Identifying Selection Snap Points For A Video
Filed May 26, 2021
Thumbnail Video Segmentation Identifying Thumbnail Locations For A Video
Filed May 26, 2021
2020
Interacting With Hierarchical Clusters Of Video Segments Using A Metadata Panel
Filed September 10, 2020
Awards and Fellowships
2019
Best Paper Honorable Mention · ACM SIGCHI Conference on Human Factors in Computing Systems
Paper Title: Implementing Multi-Touch Gestures with Touch Groups and Cross Events
2018
Best Paper · ACM SIGCHI Conference on Human Factors in Computing Systems
Paper Title: Extending Manual Drawing Practices with Artist-Centric Programming Tools
2017
Best Paper · ACM SIGCHI Conference on Human Factors in Computing Systems
Paper Title: Supporting Expressive Procedural Art Creation through Direct Manipulation
2010
Best Paper Honorable Mention · ACM SIGCHI Conference on Human Factors in Computing Systems
Paper Title: Example-Centric Programming: Integrating Web Search into the Programming Environment
2010
Best Paper Honorable Mention · ACM SIGCHI Conference on Human Factors in Computing Systems
Paper Title: What Would Other Programmers Do? Suggesting Solutions to Error Messages
2009
Best Paper Honorable Mention · ACM SIGCHI Conference on Human Factors in Computing Systems
Paper Title: Two Studies of Opportunistic Programming: Interleaving Web Foraging, Learning, and Writing Code
2008 – 2010
Stanford Graduate Fellowship · Stanford University
2005 – 2006
Stanford School of Engineering Fellowship · Stanford University
2001 – 2005
Pace Fellowship · Washington University in St. Louis
Merit-based full tuition scholarship for undergraduate study
2004
NSF Undergraduate Research Fellowship · Washington University in St. Louis
Teaching
Instructor
2011, 2010
Stanford CS147L · Introduction to Human-Computer Interaction Design (Lab)
Head instructor, course designer
Designed and taught course on Web development and design
Assistant
2008
Stanford CS376 · Research Topics in Human-Computer Interaction
Head Teaching Assistant
2007
Stanford CS147 · Introduction to Human-Computer Interaction Design
Head Teaching Assistant
First year of course redesign
Mentorship and Collaboration
Advisees
2014 – 2017
Jennifer Jacobs · PhD Student, Media Lab, Massachusetts Institute of Technology
Generals Committee, Reading and Oral Examination Committees
Thesis title: Dynamic Drawing: Broadening Practice and Participation in Procedural Art
2014 – 2016
Jan-Peter Krämer · PhD Student, RWTH Aachen University
Reading and Oral Examination Committees
Thesis title: Interacting with Code: Observations, Models, and Tools for Usable Software Development Environments
2011 – 2015
Stephen Oney · PhD Student, Carnegie Mellon University
Reading and Oral Examination Committees
Thesis title: Expressing Interactivity with States and Constraints
Interns Supervised
2016
Amanda Swearngin · PhD Student, University of Washington
2015, 2014
Jennifer Jacobs · PhD Student, Media Lab, Massachusetts Institute of Technology
2015
Sumit Gogia · MS Student, Media Lab, Massachusetts Institute of Technology
2014
Jan-Peter Krämer · PhD Student, RWTH Aachen University
2014
Victor Santos Uceta · PhD Student, Purdue University
2013
Dennis Kehrig · Diplom-Informatiker (Masters) Student, RWTH Aachen University
2013, 2012
Thomas Lieber · PhD Student, Massachusetts Institute of Technology
2013, 2011
Stephen Oney · PhD Student, Carnegie Mellon University
2012
Ian Wehrman · Postdoc Researcher, University of Texas at Austin
2012
Jonathan Diehl · PhD Student, RWTH Aachen University
2012
Robert Morris · PhD Student, Media Lab, Massachusetts Institute of Technology
2011
Juho Kim · PhD Student, Massachusetts Institute of Technology
Students Supervised
2012
Suryaveer Singh Lodha · Masters Student, UC Berkeley
2012
Ayden Ye · Masters Student, UC Berkeley
2012
Yin-Chia Yeh · Masters Student, UC Berkeley
Collaborators
Michael Bernstein (Stanford), Jan Borchers (RWTH Aachen), William Choi (Stanford), Mira Dontcheva (Adobe), Ethan Fast (Stanford), Krzysztof Gajos (Harvard), Sumit Gogia (MIT Media Lab), Philip Guo (Stanford/MIT/Rochester), Bjoern Hartmann (UC Berkeley), Jennifer Jacobs (MIT Media Lab), David Karger (MIT), Dennis Kehrig (RWTH Aachen), Juho Kim (MIT), Scott Klemmer (Stanford/UCSD), Jan-Peter Krämer (RWTH Aachen), Joel Lewenstein (Stanford), Thomas Lieber (MIT), Daniel MacDougall (Stanford), Robert Miller (MIT), Robert Morris (MIT), Brad Myers (CMU), Stephen Oney (CMU), Victor Santos Uceta (Purdue), Amanda Swearngin (University of Washington), Ian Wehrman (UT Austin), Doug Wightman (Queens University, Ontario), Noah Weiss (Stanford), Leslie Wu (Stanford), Ron Yeh (Stanford)
Invited Talks
2016
Keynote at VISSOFT 2016: IEEE Working Conference on Software Visualization · Raleigh, NC
2015
Carnegie-Mellon University Human-Computer Interaction Seminar · Pittsburg, PA
2014
Segal Design Institute Seminar at Northwestern University · Evanston, IL
2012
Berkeley Institute of Design Seminar · Berkeley, CA
2011
MIT Human-Computer Interaction Seminar · Cambridge, MA
2011
Carnegie-Mellon University Human-Computer Interaction Seminar · Pittsburg, PA
2010
UC Boulder Human-Computer Interaction Seminar · Boulder, CO
Academic Service
2016
UIST Program Committee · ACM Symposium on User Interface Software and Technology
2015
CHI Program Committee · Specific Application Areas Subcommittee, ACM Conference on Human Factors in Computing Systems
2014
UIST Program Committee · ACM Symposium on User Interface Software and Technology
2014
VL/HCC Program Committee · IEEE Symposium on Visual Languages and Human-Centric Computing
2013
WWW Research Track Program Committee · User Interfaces, Human Factors, and Smart (Mobile) Devices Track, International World Wide Web Conference
2012
UIST Program Committee · ACM Symposium on User Interface Software and Technology
2012
HCOMP Program Committee · AAAI Human Computation Workshop
2011
UIST Program Committee · ACM Symposium on User Interface Software and Technology
2009
CHI Work-in-Progress Program Committee · ACM SIGCHI Conference on Human Factors in Computing Systems
Reviewing · CHI, ToCHI, UIST, CSCW, HCOMP, WWW, IUI, Communications of the ACM, IEEE Magazine
Certifications
2012
ScrumMaster · Scrum Alliance
Open Source
Committer
Adobe Photoshop Generator · https://github.com/adobe-photoshop/generator-core
JavaScript extensibility layer for Photoshop
Brackets · https://github.com/adobe/brackets
Code editor focusing on HTML, CSS, and JS development
ConstraintJS · https://github.com/soney/ConstraintJS
JavaScript library for constraint-based definition of interactivity
Contributor
Reveal.js · https://github.com/hakimel/reveal.js
HTML Presentation Framework
Node · https://github.com/joyent/node
JavaScript runtime for servers
JSHint · https://github.com/jshint/jshint
JavaScript static analyzer and linter