MultiMedia

Squeak can present information in a variety of digital media formats (audio, video, interactivity, graphics, and animation) which provides creators various vehicles to enhance user comprehension.

Graphical Squeaking

Graphics is an area in which Squeak excels. Squeak's 2D bitmap, outline graphics and 3D modeling capabilities are amazingly flexible using the Morphic framework and the Alice 3D authoring environment.

2D Graphics

Squeak's standard graphic model is based on Dan Ingalls' original BitBlt (copying rectangular areas of the screen and off-screen pixmaps using boolean functions) which has been extended for line drawing, text display, and affine transformations: scaling, rotating, etc (see WarpBlt) for 1,2,4,8,16 and 32 bit depth. Classes are included for reading and manipulating BMP, JPEG, GIF, TIFF and PNG files, amongst others.

Video and Image Processing, is a nice framework for picture adjustments and adding filters to pictures and video. Grab it from SqueakMap and shiftclick on the picture you want to edit.

3D Graphics

The underlying vector engine for 2D outline graphics in Squeak is known as Balloon. The 3D extension of that engine is known as Balloon3D.

Alice

Alice is an authoring tool designed for people without programming experience to build interactive 3D worlds. The Stage 3 Research Group (Randy Pausch director) started the Alice project at the University of Virginia, and is continuing their work at Carnegie Mellon University's Stage3 Research.

Squeak-Alice brings the ideas and the ideals of Alice to Squeak, with the goal of introducing even more people to 3D graphics and allowing them to author in the medium. Wonderlands are the basic building blocks of Squeak-Alice and are interactive 3D worlds. To create a Wonderland, evaluate:

Wonderland new.

This creates a script editor, a control panel, and a window into the 3D world. From here you're ready to start building your world:

Screenshot of 3D Alice environment

Interactivity

Morphic

Morphic is a direct-manipulation User Interface (UI) construction kit based on display trees. It is an alternative to the traditional Model View Controller (which is still available in Squeak. (MVC). It may obsolete MVC, as Dan Ingalls says in Place in the Universe.

Basics

• Morphic Architecture
• What are Morphs?
• Important messages using Morphs

Morphic tutorials:

• Morphic Tutorials
• Fun with Morphic
• Fun with the Morphic Graphics System by John Maloney.
• Morphic for Dummies by Mark Guzdial.

Where does it come from?

Squeak Team member John Maloney originally co-developed and created Morphic, with Randy Smith, as part of the Self language-environment project at Sun Microsystems Laboratories. More readings at the Sunlabs mirror.

Self tutorial: Prototype-Based Application Construction Using SELF 4.0

Self tutorial Heading: Morphic: The SELF user interface framework

Here is a pdf version of "Morphic: The Self User Interface Framework", the original paper describing the Self version of Morphic (also available in compressed postscript). John Maloney instigated Squeak's implementation of Morphic, and wrote MethodForMakingMorphs. Morphic is part of Squeak since version 1.22 and has been enhanced with each version. Historical screenshot!

Other links

• Open Directory Morphic Toolkit page
• Programming in Morphic
• MarkGuzdial's "Squeaking" site has Morphic links.
• Getting Started also by Mark.

You can also share your created morphs with Morphic, with other squeakers.

Croquet Project

An exciting new development is the collaborative 3D environment called Croquet. The Croquet project is an effort to develop a new open source computer operating system built from the ground up to enable deep collaboration between teams of users. Croquet enables communication among users by acting as the equivalent of a broadband conferencing system built on top of a 3D user interface and a peer-to-peer network architecture. Find out more at The Croquet Consortium.

Tweak

The Tweak Project Website.

Squeak Audio

Music

Squeak has amazing sound capabilities, from playing MIDI files (via an internal synthesizer or via a MIDI-compatible external synthesizer), to generating multiple instruments with multiple voices at once (see the classes AbstractSound and FMSound, especially the class methods), and even recording sound and playing a wide variety of sampled sound files (including WAV, AIFF, and u-law files).

The Score Player

Squeak can play instrumental music stored as standard MIDI files. (MIDI is a standard developed by electronic instrument manufacturers for controlling music sythesizers.) Unlike audio files such as MP3, a MIDI file records only the performance gestures needed to perform a piece of music. For example, the sequence of key presses and releases on a synthesizer keyboard requires just a few bytes. A MIDI file is therefore quite small compared to an audio file for the same piece.

All the sounds can be generated by Squeak's built-in FM synthesis algorithm. On some platforms it is possible to route the MIDI control stream to an external MIDI synthesizer. Instruments in the Squeak-synthesizer will sound exactly the same on all platform - and you don't need a synthesizer and MIDI interface!

The Piano Roll displays notes as they are sounded If you pause the Score Player and click in the piano roll, but far from any notes, you will see a menu; try expand time. If you click on a note, you can play it or change it. If you click near a note, you can play all the notes in a vertical slice through the score at that point.

FM Sound Editor

Are you bored with Squeak's available sounds? At the bottom of the instrument pop-up menu is a command that will allow you to edit that instrument. This will open a new panel on your screen, giving you control over the time-varying parameters that determine that sound. These envelopes control the volume, pitch, modulation ratio and modulation depth used by the FM synthesis algorithm.

Each sound sample is divided into three time periods called attack, sustain and decay. This is called the "envelope" of the sound sample. The attack is played first, then the sustain is played repetitively (ie "looped") as long as the note is held. When the note is released (not looped anymore) the decay portion of the envelope is played to end the note. The duration of each of these periods can be changed by dragging the orange triangles at the top of the editor. Editing an envelope is just like editing a Squeak polygon: drag the yellow circles to move vertices, and drag the green triangles to create new vertices. Drag a vertex over its neighbor to delete it. Note that vertices on the sustain boundaries are tied together to ensure continuity when the sound loops back. To hear your sound at any time click on the keyboard, or start up the score player and edit the sound as it plays.

Here's something to try: drag the yellow vertex in the center of the sustain portion of the "mellowBrass" sound down to zero. You should be able to hear a warble, or tremolo, in the long notes. Changes made in the sound editor take immediate effect, so that if the voice you are changing is being played, you will hear the changes in real time. The changes only affect the copy of the sound being used by the score player. You can use the Sound Editor menu to save your changes to Squeak's sound library in this Squeak image.

Sampled Instruments

Squeak also supports instruments based on "sampling", or playing back recordings of real instruments. Sampled instruments can sound realistic. Unfortunately, sampled instruments have two disadvantages: they consume a lot of memory space and commercial sample libraries are tightly controlled by copyright. However, if you have access to a sample library in .AIFF format, you may want to investigate the classes LoopingSampledSound and SampledInstrument.

Speech! Speech!

Squeak includes the KlattSynthesizer, a Klatt-style cascade-parallel formant synthesizer. If you want to try some examples, just take a look at the 'examples' class method categories in the DECTalkReader and Speaker classes.

The synthesizer is linked to a system for generating cartoon faces, two of which are shown on the right. So not only can Squeak talk to you, it has a human face too (well, almost...)

Video Squeaking

Squeak also supports mpeg2 and an internal jpg compressed video format.