A Caret for Your Thoughts: Adapting Caret (Ꮖ) Navigation to Visual Editors

What if text editors supported drawing a decision tree inside of code instead of writing a sequence of if-statements; or typing nicely formatted math equations; or embedding editable graph diagrams; or embedding image editors? In general, what if code editors could embed visual editors in text? I've been exploring this question through building Polytope, a code editor prototype that combines text and visual editors.

In a text editor, we navigate through code by moving a caret (Ꮖ) using our keyboard and mouse. In Polytope, navigation has to work between both text and visuals. Can we adapt caret navigation to visual editors, to create a bridge between navigating text and visuals? In particular, can we translate index-based caret navigation into spatially-based caret navigation for visuals?

This talk is a follow up to my PX/22 talk:

let chart = ⊥; let song = ♩; // Embedding Visual Languages in Code

.

sources: https://docs.google.com/document/d/1IcSet3qw1ns6hx_Uu3vuYE5UfTtxl48oh51i9Ogrl6o/edit

https://elliot.website/editor/