September 7, 2010

Mobile UIs: is AIR + Multiscreen the solution?

Filed under: AIR,Flex,Mobile,UI/UX — Steve @ 12:08 PM

UX consultant Suzanne Ginsburg has posted a good article entitled Android & iPhone App Design: Is it twice the work? It’s a great insight into what to consider when developing an application UI for mobile devices, discussing the cross-platform differences and platform UI conventions that require consideration, including elements like device button types and placement. You have to consider your UI on anything from small phones to large tablet devices, while targeting one OS, and this means you either create separate versions of your app (e.g. Angry Birds and Angry Birds HD on iOS) or carefully optimise your UI design.

After reading the article, I wondered how well Adobe AIR could really address these considerations, especially versus tools like OpenPlug ELIPS that export native code from a Flex project; at the platform level it will remove the need to code in native languages, and arguably would be more ‘native’ than a HTML web app, but AIR won’t be able to reach every platform, and doesn’t directly address the other issues discussed; we need more than the AIR platform on its own to solve this one.

Adobe has a strategy for this  – something it terms ‘multiscreen projects’. This concept goes a long way to allowing developers to get applications onto as many devices/platforms as possible, but it’s still a tricky thing to attempt to solve.  If the tools support the idea of decoupling application logic and views, the abundance and maturity of AS3 MVC frameworks out there now should make it even easier to design an application and target different devices. Good tooling and component libraries has been one of Flex’s strengths, but it would also help to have a strong set of mobile-optimised UI components for each platform. Flex 4.5 (Hero) is the first evidence of this at Adobe’s end, but I’m hopeful we’ll see other open-source and paid component frameworks for mobile Flex being made available.

Finally, going back to Suzanne’s article, all the UX articles I’ve read so far on mobile UI development seem to concentrate on catering for different screen sizes and resolutions with a fixed layout, but part of me wonders why this can’t be partly addressed with fluid layouts. We’ve been doing this for years with HTML (and Flex), and while it can’t deal with extremes, it is possible with careful scaling and layout work to achieve a UI that can work on different resolutions and screens. It’s not ideal, but it’s nothing new either.  One of the key differences between a desktop UI and a mobile UI is windowing, so there is emphasis on a full-screen fixed layouts, but I wonder what the scope is for ‘best-fit’, flexible fullscreen UI layouts in future?

November 4, 2008

AIR FileReference and scope

Filed under: AIR,Development,Flex — Steve @ 8:34 PM

I ran into a small problem recently in my AIR application when using FileReference to initiate and monitor a download operation; no events from a FileReference object created within a method were being fired. After a quick check of the Livedoc entry I found the issue:

if the FileReference object goes out of scope, any upload or download that is not yet completed on that object is canceled upon leaving the scope. Be sure that your FileReference object remains in scope for as long as the upload or download is expected to continue.

So basically a FileReference object needs to be created in such a way that it stays in scope for the duration of your operation; if you instantiate it within a method it there’s a good chance it will go out of scope and any events you were waiting for from it will never fire. The easiest solution is to declare it outside of the method, so instead of:


private function download(event:MouseEvent):void
{
var fileRef:FileReference = new FileReference();
fileRef.addEventListener(IOErrorEvent.IO_ERROR, onIOError);
fileRef.addEventListener(Event.COMPLETE, onComplete);
//etc.
}
you need to do something like this:

private var fileRef:FileReference();
private function download(event:MouseEvent):void
{
fileRef = new FileReference();
fileRef.addEventListener(IOErrorEvent.IO_ERROR, onIOError);
fileRef.addEventListener(Event.COMPLETE, onComplete);
//etc.
}

Sorted. Afterwards I also found more info about this via a couple of useful blog articles here and here.

September 26, 2008

AIR: Rendering native system icons, Pt.1

Filed under: AIR,Development,Flex — Steve @ 2:01 PM

I was asked how I got AIR to display native file icons in a component – it’s pretty easy to do, although my method is a little convoluted to explain without posting reams of code, partly because it’s buried in a sequence of Cairngorm events/commands but also because there a couple of important issues to watch for and handle (see bottom of this entry for those). Here’s an overview:

AIR has support for retrieving the native system icons in whatever supported sizes exist. The icons are stored as a property of each File object as BitmapData, in an array, File.icon.bitmaps. Each element in the array is the same icon at different sizes, e.g. 32×32, 16×16 etc.

In order to get at an icon at a given size, you can’t rely on what sizes are available at a given element position, so you need to create a new (empty) BitmapData object at your target dimension, then iterate through File.icon.bitmaps until you hit the matching-sized BitmapData object. Once you have a match, you can put the matching data into your own BitmapData object. Here’s a brief example:

public function get32Icon():BitmapData {
var myFile:File = new File("C:/foo.txt");
var bmpData:BitmapData = new BitmapData(32, 32);
for (var i:uint = 0; i < myFile.icon.bitmaps.length; i++) {
if (myFile.icon.bitmaps[i].height == bmpData.height) {
bmpData = myFile.icon.bitmaps[i];
}
}
return bmpData;
}
Obviously you need more than the code above to handle situations where the 32×32 icon isn’t available, but that’s a basic way to grab the icon as BitmapData. At this point you could create a new Bitmap object and give it the captured data, but for my application I set the icon data back onto an Object that represents the File object (I actually used an ObjectProxy because I wanted to bind this data to an ItemRenderer later) – again this becomes important later on.

Okay, so now I have my icon data, in an object that also contains other information about the file, like its name etc. To display it in a TileList, or other component, I just use a custom ItemRenderer. I set up an image tag for the icon within the renderer:

<mx:Image width="32" height="32" source="{getIcon(data)}" />
…and then create a method in the renderer to return the icon data to the image component:
private function getIcon(data:Object):Bitmap {
var bmpData:BitmapData = new BitmapData(32, 32);
bmpData = data.icon;
var iconBmp:Bitmap = new Bitmap(bmpData);
return iconBmp;
}
Now each time the ItemRenderer has to render an item, it gets the relevant icon, the filename etc. and displays them within the TileList – easy! Here’s the result, showing the app running in XP and OSX: Files part screen grab

But there are caveats; AIR does not behave consistently on all platforms with icon data. Here are couple of the problems I encountered:

  • Performance. There seem to be some differences in execution time for file system queries in AIR. Originally I had an ArrayCollection of File objects as the DataProvider for the TileList, retrieving icon data for each one in the ItemRenderer as required. On Windows this seemed fine, but on Mac OSX it proved to be very slow, to the point where my app was unusable. I overcame this by using the Flex Profiler to see what was causing the problem, finding that the underlying get icon() execution time was very long on OSX. By grabbing the icon data once, then caching it and other key File properties into an ObjectProxy, I was able to get OSX performance almost on a par with Windows, and this also sped things up elsewhere because I was calling get icon() once, rather than per-item in the renderer. It also improves scrolling performance of the TileList because that component renders items dynamically as they are displayed. In fact you could go one step further than I did and extend UIComponent to improve render performance even more.
  • Missing icons. AIR on Windows won’t retrieve some icons, in particular for .EXE, .TMP and .INI files. These are stored in shell32.dll on XP, but for some reason AIR can’t get to them. I also found one or two similar issues in OSX. AIR on Linux using the most recent AIR beta just returns a null value for File.icon.bitmaps, so rendering native icons is currently impossible. You need to add some way of checking for a missing icon in these cases, and swop it out for an embedded one if you can; I created a temporary workaround where I parse the BitmapData for null pixel values.

The next part of this article will deal with how I got native icons to render for remote files, but I bet you can already guess how that’s done…

September 25, 2008

AIR: Creating a custom status bar

Filed under: AIR,Development,Flex — Steve @ 12:26 PM

It’s easy to create your own customised status bar for AIR applications. For my example, I wanted to be able to display a network status icon that indicates whether the application is connected – this is bound to a state set by AIR’s network service monitor, via Cairngorm.

First of all, in your application MXML file, make sure showStatusBar is set to true, add the statusBarFactory parameter and point it to your custom component:

<mx:WindowedApplication xmlns:mx="http://www.adobe.com/2006/mxml"
layout="absolute"
showStatusBar="true"
statusBarFactory="{new ClassFactory(CustomStatusBar)}">

Then just make a new MXML component called CustomStatusBar and add any elements you want to display in your new status bar. There are a couple of things AIR will be expecting from a StatusBar, most importantly the status setter and getter methods required to display status text:

<?xml version="1.0" encoding="utf-8"?>
<mx:HBox width="100%"
xmlns:mx="http://www.adobe.com/2006/mxml"
backgroundColor="#E3E1E1"
>
<mx:Script>
<![CDATA[
import mx.events.FlexEvent;
import uk.ac.warwick.filesclient.model.AppModel;
[Bindable] private var modelLocator:AppModel = AppModel.getInstance();
[Bindable] public function get status():String
{
return this.statusText.text;
}
public function set status(value:String):void
{
this.statusText.text = value;
}
private function showNetworkStatusIcon():void
{
var path:String = "";
if(modelLocator.isNetworked)
{
statusIcon.source = "greenlight.png";
} else {
statusIcon.source = "greylight.png";
}
}
]]>
</mx:Script>
<mx:Label id="statusText" paddingLeft="7" fontAntiAliasType="advanced"/>
<mx:Image id="statusIcon"
toolTip="Network status"
horizontalAlign="right"
width="100%"
height="10"
verticalAlign="middle"
render="showNetworkStatusIcon()"/>
</mx:HBox>
You can add almost anything you like in there with this technique; animations, custom text etc. Here’s a quick screen grab of my basic custom status bar with its status light on green after a bit more tweaking of the layout and styling – nothing amazing, but its useful to be able to add your own elements when required (click to enlarge);

Status bar

August 28, 2008

David Tucker’s 10 common mistakes when building AIR applications

Filed under: Adobe,AIR,Development,Flex — Steve @ 1:04 PM

David Tucker recently posted an article on the Adobe Developer Centre citing 10 common mistakes with developing AIR applications and how to avoid them. I thought I’d quickly compare David’s points against my current AIR project, a remote file-system/transfer manager:

1. Making an application platform specific

David primarily refers here to the UI/UX differences between platforms, but I also found it essential because I encountered significant performance differences with some file system operations (icon getters, mostly), which were fixed with the help of the Flex Profiler. At the moment we can’t test it on the Linux Alpha, but hopefully we’ll be able to try that soon.

2. Not including update capability in an application

We included this as soon as the framework became available and it works very well – whenever the application starts it can check for a newer version of itself, and the user has control over whether to update or not.

3. Changing the application ID after an app has been released

Oops – I ran into this early on; changing the application ID means the update framework will break, amongst other things. Changing the name of your application halfway through the project also risks confusion.

4. Not planning for offline support

Our application relies on a live connection to work, but it does use AIR’s network monitoring APIs to check for a valid connection at startup and then continually monitor connection status, and warns the user if connectivity is lost.

5. Not thinking in AIR

I didn’t find this too difficult – some things were relevant and others weren’t. There have often been moments where I’ve discovered a capability and thought “I didn’t know it could do that”; we do use the application storage directory and the user’s temp directory for file transfers though, and the File , FileStream and FileReference classes were the key to making it work.

6. Using custom chrome to create confusing interfaces

We used the standard chrome, but most of the AIR applications I’ve seen that use custom chrome have done so pretty effectively. One of the most powerful aspects of AIR is that you have very fine control over the UI. As a counter to this though, in addition to the native window chrome there are valid use-cases for having access to native system controls, like toolbars, buttons etc. – platform UI differences can make this even more acute, so I’d like the ability to use standard UI elements where appropriate.

7. Not using the seamless install badge

We implemented this early on – having an install badge makes installation a snap for most people and like the update framework it works well – installation of the AIR runtime can be managed automatically and so far the whole thing has worked fine, except when I broke it myself by not updating the right fields in the updater XML file.

8. Not encrypting sensitive data

Not relevant for this application (yet) – we don’t store any information other than the last file-space used, using a standard SharedObject. AIR has an EncryptedLocalStore for this kind of thing though.

9. Not preserving native interaction

As yet we haven’t got a complete set of keyboard interactions in place (e.g. cut, copy and paste) but some are there. The core interaction type is dragging and dropping, and Flex/AIR gives you control over the process by splitting this action into discreet event-driven stages, so providing visual feedback about whether a drag/drop is allowed can be controlled via your own logic. Something I’ve not yet overcome is how to work with internal and external drag handling at the same time – I may be wrong here but so far it looks like the external drag management only knows about drag in/out operations at the application level, not at the component level, so I need to work on how to use the external drag management to allow items to be dragged into components with the same level of control as the internal drag manager.

A more difficult problem is when a standard component doesn’t quite mimic the operation of a native system control – take the Tree for instance, which will close when its DP is refreshed – in situations like these its nearly always possible to closely replicate the native behaviour by extending or over-riding the component default, but there can be some work involved when finding out what to do. At this point I’ll thank Peter Ent and the excellent Flex Examples Blog for their invaluable resources; they saved me a lot of time.

10. Assuming performance doesn’t matter outside of the browser

In this case, the performance issues highlighted when checking across platforms also highlighted the importance of using the Flex Profiler, the net result being a five-fold increase in speed on OSX and smaller but useful increases on Windows, plus reduced memory usage. The original performance on OSX was bad enough to almost make the application unusable, but after identifying and fixing/working around the problems, the application performs similarly in Windows and OSX.

Thanks to David for writing about what to avoid – fortunately most of them we’d already come across and fixed, so that’s good!

July 18, 2008

On Dialogue Boxes…

Filed under: AIR — Steve @ 12:45 PM

I’m currently writing a few dialogue and interaction menus for my current AIR project, and the thought had occurred to me that having established a reasonable methodology for handling and displaying modal dialogues within my Cairngorm-based app, I was perhaps using them almost by default, without thinking too carefully about whether a modal dialogue was the most appropriate means of interaction. By modal in this context we mean “A state of a dialogue that requires the user to interact with the dialogue before interacting with other parts of the application or with other applications”.

At the same time, Chris and I have been talking about metadata recently (another entry to come, but the premise was that persuading users to input metadata about assets is hard to incentivise). Related to that, Chris sent me this great link to an entry by Jeff Attwood that in turns talks about an entry by Eric Lippert on how dialogue boxes are perceived by users:

* Dialog boxes are modal. But users do not think of them as “modal”, they think of them as “preventing me from getting any work done until I get rid of them.”

  • Dialog boxes almost always go away when you click the leftmost or rightmost button.
  • Dialog boxes usually say “If you want to tech the tech, you need to tech the tech with the teching tech tech. Tech the tech? Yes / No”
  • If you press one of those buttons, something happens. If you press the other one, nothing happens. Very few users want nothing to happen—in the majority of cases, whatever happens is what the user wanted to happen. Only in rare cases does something bad happen.

In short, from a user perspective, dialog boxes are impediments to productivity which provide no information. It’s like giving shocks or food pellets to monkeys when they press buttons—primates very quickly learn what gives them the good stuff and avoids the bad.

I liked that, especially the bit about “Teching the tech” – while it’s quite funny it’s also a pretty accurate reflection of my experience as a user.

This is also related closely to what Chris and I were discussing about metadata; expecting the user to fill in information that has no obvious purpose and slows down the primary task of upload/publish or whatever it is that they are trying to do, is likely to be ignored. If those fields/dialogues are modal or conditional, it’s worth thinking carefully about whether there are alternative ways to complete the operation or gather the infomation. That’s harder to do of course, and there are cases where modal dialogues should be considered appropriate, e.g. where the application is about to do something destructive like deleting or overwriting a file, but there are alternatives, like how IE and Firefox avoid breaking the flow of interaction when blocking certain actions.

June 2, 2008

Yahoo BrowserPlus and AIR

Filed under: AIR — Steve @ 8:13 PM

Writing about web page http://browserplus.yahoo.com/

Yahoo has entered the RIA domain with BrowserPlus, a technology that allows you to create functionality that bridges desktop and browser. Some reports I’ve seen say that BrowserPlus is a direct rival to AIR, but from what I’ve read on the Yahoo site I’d say it sits closer to the browser than the desktop, like Gears; applications are still browser-hosted, whereas AIR sits more evenly between the two (albeit with much stronger emphasis on applications being system-based), but I’m wondering how many more niches there are left to fill in the RIA paradigm. BrowserPlus does offer some genuinely useful capabilities that are similar to those offered by AIR, like file drag/drop and system tray capabilities so comparisons are inevitable and in some ways justified, but these are currently a limited subset. An area which I think does justify close comparison to AIR is the development ‘gateway’, simply because both technologies allow HTML/JS developers to create RIAs with their existing skill-set.

If it’s easy to detect the existence of the BrowserPlus runtime within a page, it could get some traction; on our own web applications I can see it being useful to provide users who have it installed some extra functionality, like dragging and dropping files to the browser, or page update notifications, but it would be important not to rely on it being there for core functionality, partly I think because it would be harder to persuade users to install something that isn’t so ‘tangible’ as an AIR or Webstart application. Security may also be a concern – AIR and Java applications need to be digitally signed whereas it’s not clear yet what Yahoo will do here. I guess though it will depend on how useful and ubiquitous applications built on BrowserPlus become, and Yahoo has some fairly prominent applications for it within its own portfolio (e.g. Flickr). AIR is fast approaching that stage I think – Ebay Desktop, Google Analytics AIR, Twhirl, Adobe Media Player and Snackr are now core applications for many people I know, and the new and rather lovely Acrobat.com suite (and not forgetting Aviary) will be a major driver for further adoption, especially if the the AIR version can be extended to include desktop versions of the applications. I’m sure Yahoo will have the detection aspect covered – in addition they have done something similar to JavaFX with an extensible runtime that can be added to without re-installation, so it’s really only a case of getting that important first-install.

One final thought – it seems to me that if you could combine BrowserPlus with Prism you would have something closer in concept to AIR, if not execution; a browser-based application that looks like a desktop application, with a desktop integration runtime on top.

May 9, 2008

AS Development skills sharing at Warwick

Filed under: AIR,Development,Flash,Flex — Steve @ 11:18 PM

Chris was talking today about intra-team/department skills sharing, which I think is a great idea. With regard to Flash and Flex development at Warwick we’ve recently embarked on something along these lines, with the formation of a University Flash and Flex developers group consisting of people actively using (or just interested in) Actionscript development. It started with a forum that was initially populated with ITS people (plus one or two others), but since starting to deliver some Flash courses for ITS Training I wanted to maintain contact with what attendees did with the skills they’d acquired after the course, and the forum was opened up to allow more people to contribute. We held the first ‘skills session’ a couple of weeks ago, in the Teaching Grid. It was purposely an informal gathering, consisting of developers from ITS and departments, plus lots of coffee and biscuits. The session was an opportunity for everyone to show what they’d been working on and share expertise and a mix of ongoing Flash, Flex and AIR projects were demonstrated, plus discussion on development approaches and potential applications in the future.

Overall I thought it went well (we ran out of time in the end) and although next time some structure to the session might help us cover ground more efficiently, the informality and range of projects kept it interesting, and it was especially good to share ideas with others while receiving questions and feedback. As a result of the session, I’ve been asked to present a more formal hour-long session on AIR to the CIS Team next week, which I think I’ll prepare some slides for.

Anyway, my point is that personally I found it quite fun/rewarding to find out what other people were doing or planning to do with the technology, and to see and discuss approaches/methodologies and alternatives; we should definitely do more of this kind of thing in future. Time will tell whether we maintain the Flash/Flex/AIR group sessions but I hope we can.

Warwick staff interested in Flash/Flex/AIR/AS3 development can join the developer group forum here. For students, Warwick Game Design have expertise with Flash for game development.

April 15, 2008

Cairngen

Filed under: Adobe,AIR,Development,Flex — Steve @ 11:24 PM

Writing about web page http://code.google.com/p/cairngen/

I wish I’d come across Cairngen a bit sooner, having recently just set up two Cairngorm-based AIR projects via cutting and pasting a skeleton Cairngorm project structure from Rob’s earlier examples.

Cairngen will set up a Cairngorm project structure for you (using Ant), and much more besides. For example it’s capable of auto-generating what are termed ‘sequences’, which are sets of event-command-delegate templates (plus other alternative combinations) that create the FrontController, ModelLocator and VOs required, plus all the relevant mappings and basic classes. The core templates can also be modified so that classes contain whatever boilerplate text you need. A little downtime now learning how to use Cairngen should save me work in future, and if I set up a basic project script anyone else using Flex/AIR here can should be able to get projects up and running more quickly.

April 9, 2008

AIRTour London liveblog

Filed under: Adobe,AIR,Flex — Steve @ 10:22 AM

Writing about web page http://onair.adobe.com/schedule/cities/london.php

Just some live notes from the AIRTour London event today…

Andrew Shorten
Adobe – ‘disruptive technologies’
RIA platforms – AMP, Buzzword Connect, all built on Flex/AIR
Why AIR? – occasionally-connected opportunities, efficient development, richer interfaces.
Branding on desktop – clever clothing store that can match clothes to an image dragged

Kevin Hoyt – Building AIR apps with Javascript
Showing mapping app using Yahoo Maps API
Export map view as PNG by dragging – encodes PNG on the fly, drag map straight to desktop
Using notepad app to write an AIR app
Starts with bare HTML page
Point to XML application descriptor file – template comes with SDK
changed to point to HTML file
Running AIR from Terminal to test app – runs directly
Dreamweaver demo – extension for AIR available
Writing tags like input id=”foo” type=”button” value=”save”
then writes JS script that handles event, using DOM to grab object, then runs as AIR app
AIRIntrospector – similar to Firebug
Maps AIR namespace to JS, allows AIR APIs to be called: var stream = new air.FileStream(); direct in HTML.
Now showing Aptana as IDE fr AIR/JS dev – Aptana now supports AIR
Sandboxed environment to prevent scripting hacks, adds some limitations

Mike Downey – Lighthouse programme – looking for key apps to extend reach of AIR (e.g. EBay desktop) *
Kevin Hoyt – Integrating HTML, JS and AS*
Mixing SWFs using SWFObject – then drags an image from Webkit into his Flash app window, makes a table from the BMP that contains each px colour value(!)
Uses FileStream to re-render px values as image in other application window
Basically levering the WebKit engine in AIR to mix HTML, JS and Flash in the same desktop window and all can communicate with each other.

Serge Jespers
Deploying and updating AIR applications
Self-signed applications report identity as ‘unknown’ on AIR installer, when known ico turns green
Uses any signing mechanism, Thawte etc.
Export Release Build, import certificate
Install badge on Labs
CheckForUpdate() script, parses application descriptor file for current version, compares, prompts user to upgrade

Daniel Dura – AIR APIs
Network detection, File I/0, Window chrome
Multi-window, z-ordering, utility windows (no taskbar entry), Native menus, Drag/Drop from various sources, clipboard, System Tray, notifications, Signing, Icons
..and much more!
Windowing control is very rich
File Sync/Async APIs
SQLLite included
Lots of code on Daniel’s site
Can drag a URL into browser, browser opens
Network/Socket Monitor API – URLMonitor(URLRequest);
Dock notifications

Bah, Macbook batteries don’t last long…

Jeremy Baines – from concept to TechCrunch in 5 days
Developed Alert Thingy – AIR-based social network aggregator with alerts etc, in-application commenting on peoples blogs, actually some very useful features and a good example of using AIR.

Developing Secure AIR Applications (Oliver Goldman)
Imported/loaded content executes with the Loader sandbox – does not run with application privileges by default
Sandbox bridges – opt-in on both sides, arguments are pass-by-value, allow communications between loaded content and application
Can expose function calls, but can’t pass objects
Document.write(), eval() are prohibited to protect sandbox
Suggests only loading signed content – XMLSignatureValidator API available
Implementing update functionality makes urgent updates easy should you find a bug – make updating work first

Gah, batteries really gone now…that’s all for today – excellent day, ice-cold Coke, beer, games, short sessions, direct contact with Engineering team.

Older Posts »