Squeak can present information in a variety of digital media formats (audio, video, interactivity, graphics, and animation) which provides creators various vehicles to enhance user comprehension.
Graphics is an area in which Squeak excels. Squeak's 2D bitmap, outline graphics and 3D modeling capabilities are amazingly flexible using the Morphic framework and the Alice 3D authoring environment.
Squeak's standard graphic model is based on Dan Ingalls' original BitBlt (copying rectangular areas of the screen and off-screen pixmaps using boolean functions) which has been extended for line drawing, text display, and affine transformations: scaling, rotating, etc (see WarpBlt) for 1,2,4,8,16 and 32 bit depth. Classes are included for reading and manipulating BMP, JPEG, GIF, TIFF and PNG files, amongst others.
Video and Image Processing, is a nice framework for picture adjustments and adding filters to pictures and video. Grab it from SqueakMap and shiftclick on the picture you want to edit.
Alice is an authoring tool designed for people without programming experience to build interactive 3D worlds. The Stage 3 Research Group (Randy Pausch director) started the Alice project at the University of Virginia, and is continuing their work at Carnegie Mellon University's Stage3 Research.
Squeak-Alice brings the ideas and the ideals of Alice to Squeak, with the goal of introducing even more people to 3D graphics and allowing them to author in the medium. Wonderlands are the basic building blocks of Squeak-Alice and are interactive 3D worlds. To create a Wonderland, evaluate:
This creates a script editor, a control panel, and a window into the 3D world. From here you're ready to start building your world:
Screenshot of 3D Alice environment
Where does it come from?
Squeak Team member John Maloney originally co-developed and created Morphic, with Randy Smith, as part of the Self language-environment project at Sun Microsystems Laboratories. More readings at the Sunlabs mirror.
Self tutorial: Prototype-Based Application Construction Using SELF 4.0
Self tutorial Heading: Morphic: The SELF user interface framework
Here is a pdf version of "Morphic: The Self User Interface Framework", the original paper describing the Self version of Morphic (also available in compressed postscript). John Maloney instigated Squeak's implementation of Morphic, and wrote MethodForMakingMorphs. Morphic is part of Squeak since version 1.22 and has been enhanced with each version. Historical screenshot!
You can also share your created morphs with Morphic, with other squeakers.
An exciting new development is the collaborative 3D environment called Croquet. The Croquet project is an effort to develop a new open source computer operating system built from the ground up to enable deep collaboration between teams of users. Croquet enables communication among users by acting as the equivalent of a broadband conferencing system built on top of a 3D user interface and a peer-to-peer network architecture. Find out more at The Croquet Consortium.
Squeak has amazing sound capabilities, from playing MIDI files (via an internal synthesizer or via a MIDI-compatible external synthesizer), to generating multiple instruments with multiple voices at once (see the classes AbstractSound and FMSound, especially the class methods), and even recording sound and playing a wide variety of sampled sound files (including WAV, AIFF, and u-law files).
The Score Player
Squeak can play instrumental music stored as standard MIDI files. (MIDI is a standard developed by electronic instrument manufacturers for controlling music sythesizers.) Unlike audio files such as MP3, a MIDI file records only the performance gestures needed to perform a piece of music. For example, the sequence of key presses and releases on a synthesizer keyboard requires just a few bytes. A MIDI file is therefore quite small compared to an audio file for the same piece.
The Piano Roll displays notes as they are sounded If you pause the Score Player and click in the piano roll, but far from any notes, you will see a menu; try expand time. If you click on a note, you can play it or change it. If you click near a note, you can play all the notes in a vertical slice through the score at that point.
FM Sound Editor
Each sound sample is divided into three time periods called attack, sustain and decay. This is called the "envelope" of the sound sample. The attack is played first, then the sustain is played repetitively (ie "looped") as long as the note is held. When the note is released (not looped anymore) the decay portion of the envelope is played to end the note. The duration of each of these periods can be changed by dragging the orange triangles at the top of the editor. Editing an envelope is just like editing a Squeak polygon: drag the yellow circles to move vertices, and drag the green triangles to create new vertices. Drag a vertex over its neighbor to delete it. Note that vertices on the sustain boundaries are tied together to ensure continuity when the sound loops back. To hear your sound at any time click on the keyboard, or start up the score player and edit the sound as it plays.
Here's something to try: drag the yellow vertex in the center of the sustain portion of the "mellowBrass" sound down to zero. You should be able to hear a warble, or tremolo, in the long notes. Changes made in the sound editor take immediate effect, so that if the voice you are changing is being played, you will hear the changes in real time. The changes only affect the copy of the sound being used by the score player. You can use the Sound Editor menu to save your changes to Squeak's sound library in this Squeak image.
Squeak includes the KlattSynthesizer, a Klatt-style cascade-parallel formant synthesizer. If you want to try some examples, just take a look at the 'examples' class method categories in the DECTalkReader and Speaker classes.
Squeak also supports mpeg2 and an internal jpg compressed video format.