PV252 Lecture 1 Part 1: Element hierarchy Basic building blocks of user interfaces: Elements, Views, Widgets, ... Text Button Text Input Chart Image Tabs Each Element is responsible for how a specific piece of information is rendered. // A gross oversimplification... class Button { draw(canvas) { canvas.drawRectangle(self.bounds, self.backgroundColor) canvas.drawText(self.text, self.centerPoint(), self.fontStyle) canvas.drawBitmap(self.icon, self.iconPoint()) } } 1 2 3 4 5 6 7 8 9 10 UI Elements are structured as a tree Dialog Window UI Elements are structured as a tree Dialog Window Taskbar Body Dialog Window Taskbar Body UI Elements are structured as a tree. Dialog Window Taskbar Body Icon Title Button UI Elements are structured as a tree. Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container UI Elements are structured as a tree. Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Button Button Button etc... Part 2: Basis of interactivity So far, the Element hierarchy is completely static... how does user interaction come into play? So far, the Element hierarchy is completely static... how does user interaction come into play? Event Events Events are generated (mostly) in relation to input devices or some other external "stimuli". Events Events are generated (mostly) in relation to input devices or some other external "stimuli". Event is delivered by the "platform" (browser, OS, etc.) to an Element based on its position, focus, etc. Events Events are generated (mostly) in relation to input devices or some other external "stimuli". Event is delivered by the "platform" (browser, OS, etc.) to an Element based on its position, focus, etc. A capturing Event is delivered to an Element and that Element either consumes the Event it or not. Events Events are generated (mostly) in relation to input devices or some other external "stimuli". Event is delivered by the "platform" (browser, OS, etc.) to an Element based on its position, focus, etc. A capturing Event is delivered to an Element and that Element either consumes the Event it or not. A bubbling Event is delivered to an Element and if it is not consumed by that Element, it is propagated upwards in the Element tree. Events Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Button Button Button Tab Button Tab Button Tab Panel Title Text Image Bubbling Events Listener Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Button Button Button Tab Button Tab Button Tab Panel Title Text Image Click! Bubbling Events Listener Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Button Button Button Tab Button Tab Button Tab Panel Title Text Image Click! Bubbling Events Click! Listener Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Button Button Button Tab Button Tab Button Tab Panel Title Text Image Click! Bubbling Events Click! Listener Click! Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Button Button Button Tab Button Tab Button Tab Panel Title Text Image Click! Bubbling Events Click! Listener Click! Consumed Part 3: State hierarchy So what happens if an Event is consumed? So what happens if an Event is consumed? Until this point, the vast majority of UI systems agree on this architecture, even if they sometimes have abstractions to hide it from the programmer... Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container User Interface Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Application State User Interface Observes events Dialog Window Taskbar Body Icon Title Button Tab Container Horizontal Container Application State User Interface Observes events Updates UI These are the most fundamental questions that UI frameworks/architectures are trying to answer:    Where is application state stored? How is application state updated? How is state change propagated to the user interface? Element state "Stuff that you lose if you remove an element." Position, scroll location, input field values, ... Stored by the individual UI Elements. A useful framework for thinking about state... (what goes into each category depends on the application) Element state "Stuff that you lose if you remove an element." Position, scroll location, input field values, ... Stored by the individual UI Elements. A useful framework for thinking about state... (what goes into each category depends on the application) Dialog Window Taskbar Body Icon Tab Container Element state "Stuff that you lose if you remove an element." Position, scroll location, input field values, ... Stored by the individual UI Elements. A useful framework for thinking about state... (what goes into each category depends on the application) Dialog Window Taskbar Body Icon Tab Container State StateState State State A useful framework for thinking about state... (what goes into each category depends on the application) Session state "Stuff that you lose if you close the window." E-shop cart, login credentials, navigation... Stored by the browser or by the server (and linked to browser using session cookies). Session ends when the browser says it ends (for "normal" apps, it's usually when you close the app). A useful framework for thinking about state... (what goes into each category depends on the application) Session state "Stuff that you lose if you close the window." E-shop cart, login credentials, navigation... Stored by the browser or by the server (and linked to browser using session cookies). Session ends when the browser says it ends (for "normal" apps, it's usually when you close the app). Browser [session state] A useful framework for thinking about state... (what goes into each category depends on the application) Session state "Stuff that you lose if you close the window." E-shop cart, login credentials, navigation... Stored by the browser or by the server (and linked to browser using session cookies). Session ends when the browser says it ends (for "normal" apps, it's usually when you close the app). Browser [session state] Request with session cookie A useful framework for thinking about state... (what goes into each category depends on the application) Session state "Stuff that you lose if you close the window." E-shop cart, login credentials, navigation... Stored by the browser or by the server (and linked to browser using session cookies). Session ends when the browser says it ends (for "normal" apps, it's usually when you close the app). Browser [session state] Server [session state] Request with session cookie A useful framework for thinking about state... (what goes into each category depends on the application) Session state "Stuff that you lose if you close the window." E-shop cart, login credentials, navigation... Stored by the browser or by the server (and linked to browser using session cookies). Session ends when the browser says it ends (for "normal" apps, it's usually when you close the app). Browser [session state] Server [session state] Request with session cookie State-based response A useful framework for thinking about state... (what goes into each category depends on the application) Persistent state "Stuff that you lose if the hard drive breaks down." Account information, settings, "content", ... Stored in files or databases, both within the browser (locally) or on a server (remotely). A useful framework for thinking about state... (what goes into each category depends on the application) Persistent state "Stuff that you lose if the hard drive breaks down." Account information, settings, "content", ... Stored in files or databases, both within the browser (locally) or on a server (remotely). Element state "what the UI looks like right now" Session state "what the user is doing" Persistent state "what the user has saved" Part 4: Client vs. Server Client-side rendering Server-side rendering Not rendering in the GPU sense, but "where is state turned into HTML". Client-side rendering Server-side rendering Not rendering in the GPU sense, but "where is state turned into HTML". We started here Client-side rendering Server-side rendering Not rendering in the GPU sense, but "where is state turned into HTML". We started here Then everyone wanted to do this Client-side rendering Server-side rendering Not rendering in the GPU sense, but "where is state turned into HTML". We started here Then everyone wanted to do this And now we are somewhere in between Back-end (server)Front-end (client) UI Element hierarchy Session State Persistent State Server-side rendering Back-end (server)Front-end (client) UI Element hierarchy Session State Persistent State User request Server-side rendering Back-end (server)Front-end (client) UI Element hierarchy Session State Persistent State User request HTML Response Server-side rendering Back-end (server)Front-end (client) UI Element hierarchy Session State Persistent State User request Updated UI hierarchy HTML Response Server-side rendering Back-end (server)Front-end (client) UI Element hierarchy Session State Persistent State User request Updated UI hierarchy HTML Response Server-side rendering Element state is still managed by the client and can be non-trivial, so the client is not entirely state-less. But for the most part, client is not meaningfully changing the UI hierarchy. Back-end (server) Front-end (client) UI Element hierarchy Session State Persistent State Client-side rendering Back-end (server) Front-end (client) UI Element hierarchy Session State Persistent State Client-side rendering Back-end (server) Front-end (client) UI Element hierarchy Session State Persistent State Client-side rendering API calls (REST, Graph QL, etc.) Back-end (server) Front-end (client) UI Element hierarchy Session State Persistent State Client-side rendering API calls (REST, Graph QL, etc.) Client is responsible for updating the UI hierarchy based on both the local state and the data received from the server. Server's only responsibility is serving and storing data. (Less common, but client can also store some of the persistent state) Which is "better"? Which is "better"? Any sufficiently large project will usually mix both, at least to some extent. Which is "better"? Any sufficiently large project will usually mix both, at least to some extent. Server first: you have a lot of state that needs to be precisely synchronized across many users, the UI does not need too much interactivity (request-response)... More centralized: the response is always what's in the database. Less work for clients, no need for business logic written in JavaScript. More bandwidth (sometimes, depends), more compute. Which is "better"? Any sufficiently large project will usually mix both, at least to some extent. Server first: you have a lot of state that needs to be precisely synchronized across many users, the UI does not need too much interactivity (request-response)... More centralized: the response is always what's in the database. Less work for clients, no need for business logic written in JavaScript. More bandwidth (sometimes, depends), more compute. Client first: More complex and responsive interaction patterns, reduced server costs, modular (multiple clients, one API)... Can have offline or other "native" features. More logic (generally more complex UIs, but also logic for retrieving and sending data, caching, etc.). Part 5: Managing state Model View Controller Model View Controller Model View Controller Model View Controller sends input Model View Controller Model View Controller sends input m anipulates Model View Controller Model View Controller sends input m anipulates updates Model View Controller Model View Controller sends input (sometimes) updates m anipulates updates Model View Controller Model View Controller sends input (sometimes) updates m anipulates updates Element state Model View Controller Model View Controller sends input (sometimes) updates m anipulates updates "App logic"Element state Model View Controller Model View Controller sends input (sometimes) updates m anipulates updates Session State "App logic"Element state Persistent State Model View Controller Model View Controller Multiple types of each component can interact together (e.g. one view per "entity" in a database, one controller per business process, and one view per page). Model View Controller Multiple types of each component can interact together (e.g. one view per "entity" in a database, one controller per business process, and one view per page). Views can be often relatively large (e.g. a whole page) and relatively "active" in that they generate the complete Element hierarchy for every state change. Example:  A HTML template that is filled with data. Model View Controller Multiple types of each component can interact together (e.g. one view per "entity" in a database, one controller per business process, and one view per page). Views can be often relatively large (e.g. a whole page) and relatively "active" in that they generate the complete Element hierarchy for every state change. Example:  A HTML template that is filled with data. Model is relatively "passive": it ensures data validation, integrity and persistence. Model View Controller Multiple types of each component can interact together (e.g. one view per "entity" in a database, one controller per business process, and one view per page). Views can be often relatively large (e.g. a whole page) and relatively "active" in that they generate the complete Element hierarchy for every state change. Example:  A HTML template that is filled with data. Model is relatively "passive": it ensures data validation, integrity and persistence. Controller performs updates based on UI events. Model View Controller Multiple types of each component can interact together (e.g. one view per "entity" in a database, one controller per business process, and one view per page). Views can be often relatively large (e.g. a whole page) and relatively "active" in that they generate the complete Element hierarchy for every state change. Example:  A HTML template that is filled with data. Model is relatively "passive": it ensures data validation, integrity and persistence. Controller performs updates based on UI events. But where does this "live" in a Web app? MVC, server-side rendering Model View Controller UI Element hierarchy Client Server MVC, server-side rendering Model View Controller UI Element hierarchy Client Server User request MVC, server-side rendering Model View Controller UI Element hierarchy Client Server User request HTML Response MVC, server-side rendering Model View Controller UI Element hierarchy Client Server User request HTML Response Server-first frameworks like ASP.NET (C#), Spring (Java), Django (Python), Rails (Ruby), Laravel (PHP) are usually "MVC-like". Model-View-Template? Model Template View/Controller UI Element hierarchy Client Server User request HTML Response "MVC-like": Other minor interpretations of what View and Controller stand for. MVC, client-side rendering Model View Controller Client Server Not that common... Angluar (JavaScript; mostly older versions). MVC, client-side rendering Model View Controller Client UI Element hierarchy Server Not that common... Angluar (JavaScript; mostly older versions). MVC, client-side rendering Model View Controller Client UI Element hierarchy Server API calls (REST, etc.) Not that common... Angluar (JavaScript; mostly older versions). Model View Presenter View Presenter Model Model View Presenter View Presenter Model Events Model View Presenter View Presenter Model Updates Events Model View Presenter View Presenter Model Updates Events Events Model View Presenter View Presenter Model Updates Updates Events Events Model View Presenter View Presenter Element state Model Updates Updates Events Events Model View Presenter View Presenter"App logic" Element state Model Updates Updates Events Events Model View Presenter View Presenter Session State "App logic" Element state Persistent State Model Updates Updates Events Events Model View Presenter Model View Presenter Removes the ambiguity of who updates the View. Model View Presenter Removes the ambiguity of who updates the View. The View is completely passive, all updates are done by the Presenter. Model View Presenter Removes the ambiguity of who updates the View. The View is completely passive, all updates are done by the Presenter. Presenters tend to be more tightly integrated with the View, while Controllers tend to be more tied to the Model. Often leads to smaller/more granular Presenters in MVP vs. MVC. Model View Presenter Removes the ambiguity of who updates the View. The View is completely passive, all updates are done by the Presenter. Presenters tend to be more tightly integrated with the View, while Controllers tend to be more tied to the Model. Often leads to smaller/more granular Presenters in MVP vs. MVC. Most frameworks mentioned previously are primarily classified as MVP now, but the distinction is often slightly arbitrary (one can implement both). Model View Presenter Removes the ambiguity of who updates the View. The View is completely passive, all updates are done by the Presenter. Presenters tend to be more tightly integrated with the View, while Controllers tend to be more tied to the Model. Often leads to smaller/more granular Presenters in MVP vs. MVC. Most frameworks mentioned previously are primarily classified as MVP now, but the distinction is often slightly arbitrary (one can implement both). Same considerations for server-side vs client-side rendering apply. Model View ViewModel (MVVM) View ViewModel Session State "App logic" Element state Persistent State Model Updates Data binding Events Model View ViewModel Model View ViewModel Very similar to MVP, but the relationship between the View and the ViewModel is declarative, while the relationship between the Presenter and the View is imperative. Presenter listens to the events emitted by the View and manipulates it accordingly. ViewModel does not interact with the View directly, but observes the declared properties through a special data binding layer. Model View ViewModel Very similar to MVP, but the relationship between the View and the ViewModel is declarative, while the relationship between the Presenter and the View is imperative. Presenter listens to the events emitted by the View and manipulates it accordingly. ViewModel does not interact with the View directly, but observes the declared properties through a special data binding layer. More robust to changes in the View hierarchy. Model View ViewModel Very similar to MVP, but the relationship between the View and the ViewModel is declarative, while the relationship between the Presenter and the View is imperative. Presenter listens to the events emitted by the View and manipulates it accordingly. ViewModel does not interact with the View directly, but observes the declared properties through a special data binding layer. More robust to changes in the View hierarchy. Less verbose, data binding eliminates a lot of "boilerplate" code related to events. Model View ViewModel Very similar to MVP, but the relationship between the View and the ViewModel is declarative, while the relationship between the Presenter and the View is imperative. Presenter listens to the events emitted by the View and manipulates it accordingly. ViewModel does not interact with the View directly, but observes the declared properties through a special data binding layer. More robust to changes in the View hierarchy. Less verbose, data binding eliminates a lot of "boilerplate" code related to events. Most modern JavaScript frameworks (React, Svelte, Vue.js, ...) could be classified as MVVM (but most have other mechanisms, on top of MVVM, e.g. templates, that make this less clear) We'll come back to declarative vs. imperative UI in the next lecture... Part 7: Componentbased design Problem: The UI Element hierarchy on its own is often quite hard to manage, even if we only consider "Element state" that has nothing to do with business logic. To achieve a specific style, we may need to combine a lot of Elements that need we to coordinate and thus expose unnecessary "implementation details". UI Components UI Components Encapsulates a complex, but self-contained UI Element hierarchy, making it reusable. UI Components Encapsulates a complex, but self-contained UI Element hierarchy, making it reusable. "From the outside", a Component appears as just another Element. UI Components Encapsulates a complex, but self-contained UI Element hierarchy, making it reusable. "From the outside", a Component appears as just another Element. A well-designed Component encapsulates the state of multiple Elements that would otherwise have to interact with each other and share state. Architecturally, this allows us to convert some of the "Session-like" state into pure Element state. UI Components Encapsulates a complex, but self-contained UI Element hierarchy, making it reusable. "From the outside", a Component appears as just another Element. A well-designed Component encapsulates the state of multiple Elements that would otherwise have to interact with each other and share state. Architecturally, this allows us to convert some of the "Session-like" state into pure Element state. Was hugely popularized by React (now used by most JS frameworks), but similar ideas have existed outside of Web development before. Panel A Taskbar Window Button A TabComponent Panel B Panel C Button B Button C WebComponents https://developer.mozilla.org/en-US/docs/Web/API/Web_components/ WebComponents https://developer.mozilla.org/en-US/docs/Web/API/Web_components/ A Web standard for implementing UI Components. Came after React and similar frameworks, but provides tighter integration with browsers (e.g. CSS and Event encapsulation). You can mix WebComponents and existing JS frameworks, since each Component is just a standard HTML Element. WebComponents https://developer.mozilla.org/en-US/docs/Web/API/Web_components/ A Web standard for implementing UI Components. Came after React and similar frameworks, but provides tighter integration with browsers (e.g. CSS and Event encapsulation). You can mix WebComponents and existing JS frameworks, since each Component is just a standard HTML Element. Each Component behaves just like an ordinary HTML Element with a custom tag name. WebComponents https://developer.mozilla.org/en-US/docs/Web/API/Web_components/ A Web standard for implementing UI Components. Came after React and similar frameworks, but provides tighter integration with browsers (e.g. CSS and Event encapsulation). You can mix WebComponents and existing JS frameworks, since each Component is just a standard HTML Element. Each Component behaves just like an ordinary HTML Element with a custom tag name. A Component can have a shadow DOM, i.e. a hierarchy of HTML elements that is not accessible outside of the component. WebComponents https://developer.mozilla.org/en-US/docs/Web/API/Web_components/ A Web standard for implementing UI Components. Came after React and similar frameworks, but provides tighter integration with browsers (e.g. CSS and Event encapsulation). You can mix WebComponents and existing JS frameworks, since each Component is just a standard HTML Element. Each Component behaves just like an ordinary HTML Element with a custom tag name. A Component can have a shadow DOM, i.e. a hierarchy of HTML elements that is not accessible outside of the component. The standard adds a 1 2 3 4 5 6 7 8 9 10 11 12 13 14 More about WebComponents (and components in general) on the next seminar... Takeaways User interfaces are represented as tree hierarchies of elements (widgets, views, ...). Interacting with elements triggers events. Events update application state. Managing state is one of the fundamental roles of UI frameworks. Very broadly, state can be understood as element state, session state and persistent state. Server-side vs. client-side rendering: where is the state converted into the UI hierarchy? MVC, MVP, MVVM: basic design patterns for separating state, logic and user interface. Component-based design: defining new UI elements that encapsulate non-trivial behavior. WebComponents, a modern standard for defining UI components in the browser.