Optimality Constraint Interaction in Generative Grammar. Paper read at

CurrentlyDesigned

KaiRichter1andMichaelHellenschmidt2

ComputerGraphicsCenter(ZGDV),Fraunhoferstr.5,283Darmstadt,Germany,

kai.richter@zgdv.de

2

Fraunhofer-InstituteforComputerGraphics,Fraunhoferstr.5,283Darmstadt,Germany,michael.hellenschmidt@igd.fraunhofer.de

1

Abstract.Informationtechnologyshouldbeaccessibletoallusers.Abovealldisabledpersonsinapublicenvironmentdependonsuchac-cessibilitytotoday’sinformationtechnology(e.g.cashdispensers,ticketsellingmachines,etc.).Inthisarticleweshowhowmultimodalitycanbeusedtoprovideuniversalaccessibilitytopublickiosksystemsmakingas-sistivetechnologyapplicableinthepublicspace.WeintroduceasystembasedonthegenericEMBASSIinfrastructurebymeansofwhichtermi-nalsystemscancommunicatewithpersonalizedmobiledevicesoveranamodalinformationlayer.Thosemobiledevicesprovidetheuserwithadaptedmultimodalmeansofinteraction.Weillustratedifferentdeviceexamples,speciallyadjustedtotheneedsofvisuallyimpairedandpara-plegicusers,whichhavebeenrealized.Theresultsofapsychologicalevaluationoftheapproachwillbepresented.

1Introduction

Inthepastdecadeinformationtechnologyhasevolvedfromanexpert’sdomaintoaneverydayphenomenonthathaschangedoureverydaylife.Aseverynewinventionalsoinformationtechnologyisadouble-edgedsword:itcanprovidepeoplewithnewcapabilitiesalsoasitcanconfrontthemwithproblemstheydidnothavebefore.Onegroupofpeoplehasexperiencedthisverygravely:peoplewithdisabilities.Ontheonehand,speechinterfacesallowvisuallyimpaireduserstobrowsethewebandopenupnewdimensionofinformationspaces,andquadriplegicuserscanuseadvancedinteractiondevicestocontroltheirenvironmentindependentlyofferingnewfreedomandqualityoflife.Ontheotherhand,however,atthesametimethoseblessings-eventhoughexistent-arejustnotavailableinmanycontexts,e.g.inthepublicspace.ThefastevolvingtechnologiesareaddressingthebroadmarketdefinedbythecentraltwoorfourstandarddeviationsoftheGaussiandistributionofthepopulationandnotbythosepersonswhicharenotfittingintothatscheme.

Theextentofthisproblembecomesevidentinthepublicspace,inthestreets,onthemarketplacesorinthestations,whereinformationtechnologyoccurin

termsofpublickiosksystems,informationterminals,etc.Thosemachinesaregenerallyjustnotusablebypeoplewithdisabilities.Forexample,letustakeavisuallyimpairedpersonstandinginfrontofanATMwithunlabeledsoftkeysandwithoutspeechoutput(ontheotherhand,whowantstohaveanATMconfirmingloudlyadisbursement?)oraperson,thatsitsinawheelchairandtriestoreachthebuttonsonaticket-sellingmachine.Inthepastdecadesgovernmentsandinternationalorganizationshaverecognizedtheseverityofthisproblemandanumberofpolitical,legalandstandardizationefforts(e.g.,Europeanyearofpeoplewithdisabilities[34],USAmericanRehabilitationAct,Section508[9],ISOStandards9241ff.and13407[15,16],WebAccessibilityInitiative[35])havebeenbroughtontheirwaytosupporttheimprovementofaccessibilityofinformationtechnology.Buteventhoughthoselawsandregulationscanserveasguidelinesandexpressionsofintent,theywillnotprovidesolutions.

Incaseofpublickiosksystems,howcanusersbeprovidedwithalltheassis-tanceandsupporttheydeservetointeractindependently?“Ifapersoncannotoperatetheworldasitiscurrentlydesigned[...]”[36]eithertheindividualhastochange,ortheworldhastobechanged,orwehavetobuildabridgebetweenthetwo.Itwillnotsufficeforcingterminalproviderstoequiptheirmachineswithspeech-outputandBraillelines,becausethisdoesnothelptheuserinthewheelchair.Itwillnothelptolowerthedisplayandthebuttonssothattheyareaccessibletothoseinthewheelchairbecauseitthenmightbeoutoftheareaexpectedbytheblind,andcertainlyhardforsomeonestandinguprighttobeaccessed.Further,economicandsecurityreasons(e.g.vandalismandmanipula-tion)preventprovidersfrominstallingcomplexandexpensiveadditionaldevices.Inshort,itwillnotbepossibletodesignaterminalsuitabletotheinfinitedi-versityofallusers.Basedonananalysisincludingthefeedbackonthedesiredsupportduringtheuseofautomatictellermachinesof99participants(39visu-allyimpaired,22physicallyhandicapped,and38userswithnodisabilities)weidentifiedthefollowingrequirements3:visuallyimpairedfelttimepressure,un-certaintyandinconveniencebysurroundingsoundsandotherpeople;physicallyhandicappedreportedtofeelhinderedbythephysicalaccessibilityofsuchsys-tems;interestinglyallgroupswereaffectedbytheoverallaccessibilityandtimeconstraintsinuseofsuchsystems.Apossiblesolutionsuggestedbythoseuserswastheuseofapersonalmobiledevicewhichwouldserveaspersonalassistantandwhichwouldprovideadditionalcustomizationandinteractionsupport.Ratherthaninstallingassistivetechnologyintotheenvironmentwethere-foreproposetodevelopmobileandpersonalizedassistantsthatcanbecarriedaroundbytheuserandthatcanbeusedtointeractwiththeenvironmentascanbeseeninfigure1.Suchassistantscanbeminiaturizedandcustomizedde-vicesequippedwithexactlythoseinputandoutputdevicesrequiredbytheuser.Ambientinformationtechnologythuswouldonlyhavetoprovideaninterfaceforsuchassistantsratherthanforeseeingeverypossiblecombinationofrequire-ments.Personalandprivatedataalsoasstoredpreferencesandhabitswillbe

3

Thework,wepresenthere,reportstheresultsoftheresearchprojectEMBASSI[13],whichhasbeenfundedbytheGermanMinistryofEducationandResearch.

Fig.1.AparaplegicpersoninteractionwithanATMthroughhispersonalmobiledevice.

availabletothisassistantinordertomakeinteractionmoreefficientthroughadaptation.Thisoffersthesymbiosisbetweenbothpersonalandadaptiveassis-tivetechnology(P-ATandA-AT[36]).

Thearticleisorganizedasfollows:inthefollowingsectionthemotivationfortheuseofmultimodalityinuniversallyaccessiblesystemswillbegiven.Thentheneedforadaptationinsuchsystemswillbeexplainedandageneralclassifi-cationschemeforthelevelsofsuchadaptablesystemswillbeintroduced.Thenabriefintroductionintoamultimodalarchitecture,theEMBASSIplatform,andalanguagefortheamodaldescriptionofuserinterfaceswillbegiven.Inthesub-sequentpartanumberofmobiledeviceswhichwererealizedwithintheprojectwillbedescribedandtheuserstudyevaluatingthesedeviceswillbepresentedanddiscussed.Finallyasummaryandoutlookforfutureworkwillsumuptheworkpresentedhere.

2AccessibilitythroughMultimodality

Wethink,thatuserswithspecialneeds(e.g.physicallyimpaired,cognitiveim-pairedandelderlypersons[4,24,29])canbesupportedinthescopeofauser-andtask-adaptablesystemratherthanbyseparateaccessibilitytools(see[31]forasimilarviewpoint).Wewilladaptthedefinitionofanaccessiblesystemasasystemthatis“abletobeusedeffectivelybyindividualseitherdirectlyorwithassistivetechnologiesthattheywillhavewiththemandcanusewhentheyencountertheenvironment,device,orsystem”[36].Wewillfurtherproposeaclassificationschemeforadaptablesystemcharacteristicsbasedonthestages

ofhuman-computerinteractionbyNorman[23]andwewillshowhowwehaveappliedthisclassificationfortheconceptualizationofoursystem.2.1

UniversalAccessibility

Accessibilityisaterm,whichistraditionallyassociatedwithelderlypeopleandpersonswithdisabilitiesregardingtheefforttheyhavetotakeinordertoaccessanduseinformationtechnology[32].AshasbeenemphasizedbyStephanidis[31,33]associatingaccessibilityprimarilytouserswithspecialneedsisshortsighted.Duetothetechnologicaldevelopmentandthesuccessiveintrusionofinformationtechnologiesintoeverydaylife,“therangeofthepopulationwhichmaygraduallybeconfrontedwithaccessibilityproblemsextendsbeyondthepopulationofdisabledandelderlyusers”[31].Beingaccessiblerequiresthatasystemisabletoadapttotheusersneeds,tothetaskscopeandcontext,andtothetechnicalplatformused.Butmorethanthat:UniversalAccessibilityimpliesthatsupportforuserswithspecialneedsarenotregardedasorthogonaltotheapplicationbutratherpartofthesystemitself.Userswithdisabilitiesarenotconsideredasadistinctclassofusers,butratheraspartofthecontinuumofhumandiversity[30].AccordingtoNorman[23]4asystemthatwantstoprovideuniversalaccessibilityhastoreducethedistancesbetweentheuser’sintentionandtheeffectsontheworld.2.2

ConceptualizationoftheAdaptableSystem

WeusedNorman’ssevenstagesofactiontodeterminetherelatedsystemcharac-teristicsthatcanbeadaptedtotheuser’sneedsinordertoincreaseaccessibility.Adaptationmethodscaneitheraddressonespecificsystemcharacteristic(e.g.ContextHelp)orcontrolseveralcharacteristicsincombination(e.g.MemorysupportandAutomation).

1.Goal:Theuserhastodeterminehowtheworldshouldlooklikeasresult.

–Memorysupport:Remindinguserofscheduledorpreviousgoals.Canrangefromnotexistenttohighlydominant,reducinginteractiontoalistofpossiblegoals(e.g.MicrosoftWindowsXP:“recentprograms”)2.Execution:Theuserhastotakethestepsnecessaryforthechosengoal.(a)Intentiontoact:Theuserhastodetermineifthesystemisabletohelp

reachingthegoal.

–Contexthelp:Thesystemcaninferthepossibleintentionoftheuserbymeansofthecontextofuseandmaygiveadvicesorremainstatic.–Adaptivehelp:Givinginformationaboutpossibleactionsindifferentbreadthanddepthandadaptedcontent.

–Modularization:Onecomplexapplicationcanmimicmodularappli-cationsinordertoreducecomplexity.

4

Normancallsthedistancebetweentherealstateoftheworldandtheusersper-ception“GulfofEvaluation”andthedistancebetweentheusersintentionandtheeffectsontheworldthe“GulfofExecution”.

(b)Sequenceofactions:Theuserhastodeterminewhatstepsarenecessary

inwhichsequence

–Guidance:Guidanceoftheuserthroughthewholeinteractionprocess.–Inputpacingandstructure:Dialogstructurecanbeadaptedtotheneedsandcapacitiesoftheuserandhiscontext(e.g.segmentationintosmallersteps).

(c)Executionofactions:Theuserhastoinitiatethephysicalprocesses

necessarytoperformtheaction

–Inputmodality:Theuserspreferredinputmodality(mightdependonthecontext(e.g.drivingrequiresspeechinput)orondisabilities).–Inputdevice:Availableinputdevices(e.g.providevirtualkeyboardifnoHWkeyboardavailable).Propertiesofdevices(e.g.delayfordouble-click)

–Automation:e.gbyshortcutsormacrostoreduceinteractioneffort

3.Evaluation:Comparisonoftheresultsofuser’sactionstohisinitialgoal.(a)Perceivingthestateoftheworld:Theuserhastoperceivethestateof

thesystemandhiscontext.

–Outputmodalityanddevices:Theuserspreferredinputmodality,thismightdependonthecontext(e.g.drivingrequiresspeechinput)orondisabilitiesandavailableinputdevices.

–Outputpacingandstructure:Dialogstructurecanbeadaptedtotheneedsandcapacitiesoftheuserandhiscontext(e.g.segmentationintosmallersteps).

(b)Interpretingtheperception:Theuserhastointerpretthestatewith

respecttotheinteractiongoal.

–Statusfeedback:Givingsystemfeedbackappropriatetotheuserneeds(e.g.moredetailedexplanationsforanexpert,recoveryhintsforthenovice,acoustictooltipstotheblind)

(c)Evaluationofinterpretations:Theuserhastoinitiatethephysicalprocesses

necessarytoperformtheactionTwostagesweidentifiedascrucial:theperceptionofthestateoftheworld(3a)andtheexecutionofactions(2c).Thesestagesmarktheborderbetweenuserandworldintheactioncycle.Withoutsupporttothosestagesabovealluserswithdisabilitieswillbeisolatedmakingobsoleteanyotheradaptation.AccordingtoOviatt[25]5wedecidetoallowtheusertoselectthesetofsuitablemodesforinteraction.

Supportingusersoveralargerspanofactionstagesisachallengingtask.Sothesecondmechanismincludedinoursystemisacombinedone.Assistanceinfindingandmemorizinggoals,sequencingandautomatingactionscanbeimportantespeciallyforuserswithphysicalhandicapsasthiswillreducethementalandphysicaleffortofinteraction.

Basedontherequirementsandtheclassificationofadaptablesystemprop-ertieswehaveidentifiedthefollowingmechanismsasrelevanttosystem:

5

Oviattpointsoutthat“multimodalinterfaceshavethepotentialtoaccommodateabroaderrangeofusersthanthetraditionalinterfaces”

–Adaptationofinputandoutputmodality:adaptinputandoutputmodalitytotheusersneeds,allowmultiplemodalitiesatthesametime.

–Adaptationofinputandoutputdevices:adaptinputandoutputdevicestotheusersneeds,supportdifferentconfigurations,andallowadaptationofdeviceproperties.

–Combinedcontext-specificsupportofgoalsetting,sequencingandautoma-tion.

3UsageScenario

Inordertoillustratetheapplicationofthetechnologicalplatformintroducedinthesubsequentsectionswewillgiveabriefoutlineofanexampleusagescenarioasitalsohasbeenevaluatedinthefinalusertests.Thiscanalsobeappliedtootherdomainsandterminaltypes,asthereareATM,andticketsellingmachines.Inourscenarioaphysicallyhandicappedpersonuseshermobileassistantathometoplanthenextshopping.Shecaninteractwiththedevicewithherchin(seefigure5)andaspecialspeechinputallowingvocalicutterancesassheisnotabletoarticulateclearlyandonlycanmoveherhead.Shefillsinrequiredgoodsintoalistofferedbytheshoppinglistagentlocatedonthemobiledevice.Thisshoppinglistagentalsosuggestsherfavoritegoodsandallowshertoreusepreviousshoppinglistsstoredonthedevice.Whenhavingcompletedthelist,sheleaveshomeandgoestothecityusingherwheelchair.Sheapproachesashoppingterminalinthemallwhichallowstoordergoodswhichthencanbedeliveredhomeorpickedupafterwards.Withouthelpshewouldnotbeabletoaccessthisterminalastheuserinterfaceisnotaccessibleforaseatedperson.Hermobiledevicenotifiesheroftheavailabilityoftheterminalandthatshenowcansubmitthelist.Sheconfirms,andthelistissubmitted.Theregularuserinterfaceoftheterminalgetslockedforotherusersforexclusiveusebythemobiledevice.Theusercontinuestocontroltheterminalwiththechininputoverhermobiledeviceandcompletesshoppingbyenteringdeliverytimeandsoforth.Thisscenarioshowshowtheindividuallyconfiguredmobiledeviceallowstheusertointeractwithherenvironmentindependentlyandhowadditionalassistancealleviatesandpersonalizesinteraction.

4TheEMBASSIPlatform

TheEMBASSIlayeredinteractionarchitecture,theagentplatformandthemul-timodaluserinterfaceconceptwillbedescribedinthefollowingsections.Themainrequirements(seethepreviouschapters)were:theabilitytoadapttotheusersinteractionneedsandcapacities,thedynamicintegrationoffunctionalitydependingonthetaskandcontextofuseandplatformanddeviceindependence.Currentlyassistivetechnology(e.g.Braillelines,speechrecognition,etc.)islinkedstaticallytothedevicesitcanoperateon(e.g.wheelchair,PC,etc.).Thisisnotpossibleinthepublicspacewheretheconfigurationofdevicesintheenvironmentisad-hocandnotforeseeable.Thereforeaplatformhasbeen

Fig.2.TheEMBASSIplatformasithasbeenrealizedforseveralpersonalmobilede-vices.Theplatformisorganizedinanumberoflevelswherecomponentscanoffertheirservices.Whenbeingconnectedtwoplatformscanthereforebejoineddynamically.

implementedonwhichassistivetechnologycouldbeinstalledandwhichcouldbeconnecteddynamicallytoanyinfrastructureintheuser’senvironment.Themobiledevicethereforeservesasmediatortothetechnicalenvironment,suchaskiosksystems,informationservices,orlocalnetworkinfrastructures.Ontheusersidethedeviceexhibitsacustomizeduserinterfacewithspecialinterac-tiondevicesonthemobiledevice.Meanwhiletheinteractionwiththeterminalitselfremainsunchangedandthusaccesstoterminalsispossibleinthetradi-tionalwayaswellasthroughasecondinvisibleinteractionchanneldesignedforusersofsuchmobileassistants.TheEMBASSIinteractionarchitecture[13,17]definescertainlevelsofabstractioneachofwhichholdscomponentsofacertainfunctionality(seefigure2).Singleuserevents,suchasspeechinputorgestureoruserinterfaceinteractionsareidentifiedbysensorcomponentsandjoinedinthefusionlevelintomultimodalinputclusters.Thedialogmanagementtrans-latestheinputintogoalsandhandsthemontothelevelsbelowwhicharetryingtoachievethesegoalsbyusingtheavailablecontextknowledgeandde-viceinfrastructure.Thislayeredarchitectureallowsthedynamicconfigurationofmodalities,functionsanddevices,aseachnewcomponentonlyhastoregisteronthelevelsitwillworkon,andthereforediffersfromotherapproacheslike,e.g.theOpenAgentArchitecture[19].Singleuserinputeventsfromdifferentmodalitiesaremergedtoamodalinformationbeforefurtherprocessing.Thus,oneabstractinterfacemayhaveseveralpresentations,whicharegeneratedbydifferentrenderingcomponents.Thedialogmanagementcomponentcancom-

municateinanamodalwaywithseveralassistantsofferingdifferentservicestotheuser.

Theframeworkisrealizedasaplatformofautonomoussoftwareagentsim-plementedinJava.Softwareagentscanbeconsideredasautonomousmoduleswhichcanactindependentlyandwhichthereforeareasuitableconceptforadynamicarchitectures.Theplatformallowsthedynamicregistrationtoaplat-form’srouter[8]anditsupportscommunicationofdifferentsoftwaremodulesso-calledagentseitherdirectlyorthroughnamedbusses.Severalsuchrouterscanbejoinedformingadynamicnetworkofagentplatformswhichcancommu-nicateanonymouslyoverthedefinedbusses.AgentcommunicationisperformedoverTCP/IPinanextendedversionoftheKnowledgeQueryandManipulationLanguage(KQML)[18].

OntheterminalalsoasonanymobiledeviceusingtheEMBASSIplatformsuchanagentnetworkisinstalledandrunning.Whenapproachingaterminal,themobiledevicecandetectthisterminal(whichadvertisesitselfoverbroad-casts)anditcanestablishanetworkconnectionjoiningbothplatforms.Severalassistantcomponentscanbeavailableeitherontheterminaloronthemobilede-viceoronanyotheragentplatformthatisconnected.Theyareadvertisingtheirservicesonalocalcontextmanagerthatservesasalocalstorageandservicebrokeragedevice.Theseservicescanbediscoveredandusedbyanyagentthatisconnectedtotheplatform.TheplatformthereforesupportsthefullydynamicconfigurationofagentsbasedontheoverallEMBASSItopology.

AterminalwhichissituatedsomewhereinthepublicspaceconstantlyemitsanXMLdescriptionofitsservicesandphysicallocationandotherrelevantinfor-mation.Oncethemobiledevicesgottheservicedescriptionstheyarepresentedtotheuser.Thustheusercantrackandselectavailableterminalsandtheirservicesandcanstartaconnection.Alternativelyservicescanbescheduledbytheuseralarminghimwhenapproachingaterminalofacertainclass.WhentheuserchoosestoconnectaterminalaTCP-connectionisestablishedandthehost-router(theterminallocatedrouter)setsupanewsessionallowingthecommunicationbetweenthemobiledeviceandtheterminal.Eachconnectiontoamobiledeviceisencapsulatedinasession,thusaterminal(andtheagentsrunningonit)isabletomanagemorethanoneservicerequestatthesametime.Alsosecurityofpersonaldataisguaranteed.

5TheUserInterfaceConcept

Aswehaverecognizedinthecollectionofrequirements,adaptationofmodal-ities,dynamicconfigurationandtechnologyindependencearenecessaryforanaccessiblesystem.Theuserinterfaceconceptintroducedheremainlyaddressesthefirstandthethirdclaim.Theabstractuserinterfacedefinitionlanguageusedinthemultimodalarchitectureallowspresentationofuserinterfacesindifferentmodalitiesinparallel.TheusageofopenandplatformindependenttechnologieslikeXMLandJava)allowsthedeploymenttovirtuallyeveryplatform.

Fig.3.AsimplifiedXUIcodeextractdescribinganinputcontrolwhichcanbepre-sentedondifferentoutputmodulesasshownontherightside,inHTMLastext-inputfield,onaspeechoutputasprompt,andonaBrailleline.

Asdescribedabovethemobiledevicesdevelopedsupportdifferentmulti-modalinteractiondevices(e.g.tactiledisplay,speechoutput,orspeechinput).Giventhefactthatthereisnorestrictiononthepotentialdeviceconfigurationssupportingdifferentmodalities,terminalsmustpublishtheirinterfaceinforma-tioninanamodalway.Thisabstractandamodaluserinterfacedescriptionistransportedtotheremote,mobileclientsinordertoallowtheusertointeractwith.Theinterfacedescriptionwedevelopedhasthefollowingcharacteristics:–isisassmallaspossible,becauseitistransportedoveranetworkfromtheterminalplatformtothemobileclients.

–itisadaptableandplatformindependent,notonlyontheterminalside,butalsoontheremotedevices.

–thedescriptionofadditionalinformationbesidestheappearanceofthegraph-icaluserinterfaceispossible.

–anditisnotlimitedtoonespecificmodality,butallowsthemappingonothersingularormultiplemodalities.AnXML-basedinterfacedescriptionlanguageseemedtoaccomplishbesttheaffordancesofsize,platformindependenceandnetworkcompatibility.SeveralapproachesfordefininguserinterfaceinXMLsyntaxareavailabletodate,e.g.,theUserInterfaceMark-upLanguage(UIML)[1],theXML-basedUserInterfaceLanguage(XUL)[38],theAdaptableUserInterfaceTechnology(AUIT)[10]andthemodelbasedeXtensibleInterfaceModelingLanguage(XIML)[6].XULisdesignedfordefininguserinterfacesforgraphicalinterfaces,AUITisboundtoweb-basedsevertechnology,andXIMLispowerfulbutfittingbestforthemodel-baseduserinterfacedevelopmentofcompletenewapplications.AlthoughtheUIMLspecificationintroducestheconceptofdevicefamiliestoprovidelevelsof

Fig.4.Themulti-modalinput/outputoutlinedbytheexampleofthevariousinputandoutputmodalitiesofferedforthevisuallyimpaired(speechinput,speechoutput,keyboard,Braille,tactiledisplay).

abstraction,itdoesnotdefinerulestotransformamodaldescriptiontodifferentrepresentations.WedecidedtoadopttheXForms[5],arecommendationoftheWorldWideWebConsortium(W3C).XFormsdefinesagenericdescriptionofinterfaceelementsandthusallowsastrictseparationofpresentation,modelanddata,whichturnedouttobeapowerfulfeaturefortheapplicationinanamodalinterfacedefinition.FurtherthecompliancetootherXMLstandardsandrecommendationsseemstoguaranteethesustainabilityofthisapproach.Basedonthisconceptwedevelopedagenerictoolkit,whichdefinesasetofmodalityindependentcontrols.

WedesignedtheeXtensibleUserInterfaceDescription(XUI)asanewhostlanguageforXFormsinordertomeetourrequirementsaccordingtoplatformindependency,multimodality,adaptability,andscalability.OriginallyXFormshasbeendesignedtobepartofXHTMLdocuments.However,thespecificationalsoallowsXFormstobehostedbyotherXML-compliantdocumenttypes.OurextensionoftheXFormsmodelconsistsofauserinterfacedescriptioncalledXUI,whichservesasahostforamodifiedXFormsapplication.Figure3illustratesanexample.Amoredetailedoverviewaboutourinterfacedescriptionconceptshouldnotbeinthescopeofthispaper(formoredetailspleasereferto[27]).

6MultimodalInteractionPipeline

Asdifferentmobiledevicesaresupportingdifferentconfigurationsofinputandoutputdevicestheinterfacedescriptionisdistributedbyaterminalinanamodalrepresentation,theXUIdescriptionoftheterminal’suserinterface.Thefilteringandrenderingcomponentsonthemobiledevicesusethisabstractinformationtogeneratemodality-specificrepresentations.Inputandoutputofeachinterac-tioncomponentisrelatedtothesameunderlyingdatamodeldeliveredwithin

theXUIdescription.Everychangeinthedatamodeliscommunicatedtothefusioncomponent,whichevaluatesthisoutputandincaseofsuccessfulevalua-tionsynchronizestheothercomponentsandsendstheeventtotheapplication(seefigure4).Inthecaseofrelatedeventsthefusioncomponentcanintegratethem,otherwisetheyareputinaqueue.Thisfusion(polymodalinputandout-putcomponent(PMIO))analysesandmodifiesoutgoinginterfacedescriptionsanddecideswhichrenderingcomponentswillreceivetheinterfacedescription.ThePMIOisextensible,inthefashionthatinputoroutputdevicescanpluginadaptationmoduleswhichwillusetheinformationfromthedescriptiontoaccommodatetheinteraction(e.g.changingthestep-modeinaonebuttonI/O,seefigure5).

Thusitispossibletocontrolterminalservices,whicharerepresentede.g.byanacousticinterfaceandatactiledisplayinacombinedmultimodalway.ToguaranteeahighflexibilityofcombinationsofmodalitiesthePMIOconsistsofdifferentmodules.ThismeansthatoneachdescribedmobiledevicedescribedbelowthesamePMIOcanbeused,howeverindifferentconfigurationsaccord-ingtothedifferentsupportedmodalities(allowingtheextractionofinformationfromtheinterfacetoconfigureI/Ocomponents,transformationsoftherepre-sentationandvalidation).Ofcourseapuregraphicalrepresentationthatreflectsthegraphicaluserinterfaceoftheterminalisalsopossibleandconsequentlytheamountofpossibleusergroupsenclosesalsothenotdisabledpeople.

7PersonalMobileDevices

Theadvantagesoftheapproachtousemobiledevicesareobvious:Accessbymobiledevicesbroadenstheamountofusergroupswithoutanyimpairmentoftheconventionalusagepossibilitiesofcommonterminals,aswehavearguedabove.Useradaptationthroughpersonalizedassistance[26]makesitpossibletosupportuserprofiles[12]toshorteninteractiontime.Theusercanforinstancecompletebanktransactionsorshoppinglistssafelyofflineathomeandsubmitthemtoaterminallateroninordertoreceivegoodsorinformationathandwhenheisonlocation.Thisdisburdenstheassisteduserfromphysicalandmentalload.Inthefollowingabriefoutlineofthepersonalmobiledevicesdevelopedwillbegivenbeforewepresenttheresultsoftheuserstudies.Personalmobiledevicesforfourgroupsofusersweredeveloped:apalm-sizedinteractiondevicefornon-disabledusers,amobiledeviceforvisuallyimpaired,anotherforphys-icallyhandicappedandoneforelderlypeople.Assistivetechnology(hardwareandsoftware)mainlyisavailableforthePCplatformthereforeportablePCswereusedfortheevaluation.

Themobiledevicefornon-disableduserswasapalm-sizedCompaqiPAQPocketPCdevicethatwasequippedwithaJava-basedHTML-front-end,whichallowstointeractwiththeterminalthroughHTML-pagesthatweredynamicallygeneratedoutoftheXUIdescription.

Fig.5.a)Avisuallyimpairedpersonusingthegraphicaluserinterfaceofaterminalwiththehelpofatactiledisplay(righthand)andthecontextinformationgivenbyBraille.b)Aparaplegicpersoninteractingwiththeterminalservicesthroughanas-sistiveinputdeviceoperatedwiththechin.Thefewoperationaretranslatedbythecontrolunitattachedtothetopofthecomputerscreen,allowingdirectionalinput,interactionwithavirtualkeyboardandstandardinputslike’enter’,etc.

Themobiledeviceforvisuallyimpaired(figure5showsavisuallyimpaireduserinteractingwithaprototypeduringtheevaluationstudies)offersthefol-lowingmultimodalinteractionpossibilities:

–Braillekeyboardorregularkeyboardforinput

–Braillelinetogiveinformationaboutthecurrentfocusoftheuserinterface(thecurrentfocuscanbechangedby“tabbing”throughtheinterface)

–Atactiledisplaythatpresentsalow-resolutionimpressionofthegraphicinformation.Technicallythisdisplayconsistsofasquarematrixofpinsop-eratedbyPiezo-technique-sameasaBrailleline-givingatactile“blackandwhite”image.

–Threedifferentvoiceoutputsfortheinformationoftheuser:(a)onethatreadsthecaptionofthecontrol,thathasthecurrentfocus;(b)onethatgiveshinttextsincasetheuserpressthe“help”button;and(c)onethatgivesproblemreports.Thecombinationof“Feel”and“Hear”turnedouttoprovidethevisuallyhand-icappeduserwiththefullcontroloftheterminal’shandlingpossibilities.Eventhoughtheseearlyprototypeswerehardlyportable,alternativePiezo-modulesareunderevaluationandBrailleextensionstomobilephones(e.g.NokiaCom-municator)arealreadyavailable.

Inmanycasesquadriplegicpersonsarenotabletoarticulateclearlyandtheirmotorcontrolcanberestrictedtoonlyfewpartsofthebody.Thusthemobiledeviceforphysicallyhandicappedassistthespecialneedsofthisusergroupbyofferingthefollowingmultimodalandassistancefunctionalities:

–Asmallcontrolunit,whichmapsaninputdevicewithonlytwoorthreedegreesoffreedomtodifferentoperations,like“enter”,“navigation”,or

“delete”.Thecurrentlyavailableinputoptionishighlightedandtheuseronlyhastoacceptorwaituntilthenextoptionishighlighted.

–Voiceinputispossibleforcertainfunctions,suchas“next”,“ok”or“abort”byadaptablesoundsratherthanwords,because-asoutlinedabove-alsothevocalexpressionabilitiescanberestricted.

–Avirtualkeyboardincombinationwithasentencecompletionsoftware,basedonheuristicevaluationofpersonaltexts[3].Thus,physicallyhandicappedpeopleareassistedbyvoiceinput,graphicaluserinterfacesandtherecognitionofsmallestgestureexpressionsgivenbymouthorsimplehandmovements.

ThemobiledeviceforelderlypeoplewasanAcerTabletPCthatwasequippedwiththeEMBASSIplatformandaspeechI/Ocomponentinadditiontoaspecialgraphicalrenderingcomponent.

8EvaluationStudies

PsychologicaladviceandevaluationhasaccompaniedthedevelopmentoftheEMBASSIproject.Severalevaluationstudieshavebeenconductedduringtheproject[7,28].Thesystemevaluatedconsistedofthemobileassistantforphys-icallyhandicappedandthedeviceforelderlypersonsrunningtheEMBASSIplatform,whichallowedtheusertointeractwithademonstrationterminal(ashoppingterminal,offeringgoodsthatcouldbedeliveredhome),providingin-teractionsupportandinteractionadaptationassistanceasmotivatedabove.Themobiledevicesprovidedfurtherassistancethroughcontextualpersonalizationbymeansofashoppinglistassistantwhichcouldbeusedtocompleteshoppinglistslocallyonthemobiledeviceandsubmittherequestwhenusingtheterminal,relievingtheuserfromtimepressureaninconvenientinteractioninthepublic.Previouslypurchaseswererecordedandapreferencelistcomputedbasedonthisdatasupportingtheusermentally[26].TheevaluationstrategyaddressedthreemainaspectsoftheDINENISO9241/11[15]definitionofusability,asthereareeffectiveness,efficiency,andacceptance.Thestudyhadtoanswerthefollowingquestions:

1.Question1:Doesthesystemallowphysicallydisabledpersonstointeractwithterminalswiththesameeaseandqualityasnon-handicappedpersons?2.Question2:Dopersonsprofitfromtheuseoftheassistancethroughper-sonalization?

3.Question3:Howistheusersoverallimpressionandacceptanceofthesys-tem?8.1

Methods

Theexperimentaldesignwassetupasnesteddesignwithtwousergroups(phys-icallyhandicapped,PHvs.non-disabled,ND)toaddressthefirstquestion.TheNDgroupwasfurtherdividedintotwogroupseitherusingtheterminaldirectly

withPhysicallyhandi-cappedwithmobile

device

CPLACCCPLACCCPLACClist1100100100100100100list273659010092buy199911009410098buy21009194100100

Table1.Completeness(CPL)andAccuracy(ACC)oflistcompletion(list)andpur-chasingthelist(buy)forallthreeconditionsofthetest.

(%)Non-disabledwith-Non-disabledoutmobiledevicemobiledevice

(ND1)orthesecondmobiledevice(fortheelderly,ND2)inordertoaddress

thesecondquestion.Asdependentvariables,effectivenesswasmeasuredbythedegreeoftaskcompletion,andthedeviationfromtheoptimalinteractionpath;efficiencywasmeasuredbythetimeandinteractionstepsneededtocompletethetask,andthesubjectivetaskload(NASATLX)[11];finallyacceptancewasmeasuredwithanine-scaleusabilityquestionnairedesignedforthisstudy.Sixpersonswithstrongphysicalimpairments(2female/4male)andtwogroupsoftenpersonswithoutdisabilities(4female/6male,each)participatedinthestudy.Theaverageagewasabout31years,experiencewithtechnicalsystemswassignificantlyhigherforthenon-handicappedusers.

Participantsoftheexperimentalconditionwereaskedtocompletetwoentiretaskcycles,eachofwhichconsistedoffillinginashoppinglistofflinebymeansoftheshoppinglistagent,thentogototheterminalandcompletethepurchasebysubmittingthelisttotheterminal.Thelistinbothiterationsdifferedforallbuttwoitemswhichwerethesameforbothlists.Twocycleswereneededtohaveaminimalhistoryeffectofthepersonalizationalgorithmsintheshoppinglistagent,offeringpreferredgoodstotheuser.Inthesubsequentsectiontheresultsoftheevaluationstudywillbepresented,orderedbythequestionsoutlinedabove.8.2

Results

Question1:Forthelistcompletiontask,datashowednodifferenceinaccuracy(ACC)andcompleteness(CPL)betweenuserswithorwithoutmobiledevice(allgroupsreachedabout100%ACCandCPLforthefirstiteration),butuserswithoutpersonalassistantweresignificantlylessaccurateintheseconditerationthaninthefirst(U-Test:p¡.002),andalsolessaccurateandcompletethanuserswithmobileassistance(U-Test:p¡.002),whiletherewasnosignificantdifferencebetweenbothiterationsforbothusergroupswithmobiledevices(seetable2).Forthepurchaseitselftheshoppinglistassistancealsoimprovedperformanceforphysicallydisableduserssignificantly(U-Test:p=.046)comparedtouserswithoutmobiledevice.

Question2:Inordertoidentifythebenefituserscangetfromthepersonal-izationthroughadaptivelistsandthehistoryeffectontheshoppinglist,savings

Non-Non-disabledwithPhysicallyhandi-disabledmobiledevicecappedwithmobile

withoutdevicemobiledevice

timesaving(min)LELEAELEAElist1.72.0-1.13,1buy0.61.41.1-0.53,8reductionofinteractionstepsLELEAELEAElist5.52.9-6.3-3.8buy2.812.67.9-2.58,7

Table2.Timesavingandreductionofinteractioneffortforthethreeconditions,partitionedintolearningeffect(LE)andassistanceeffect(AE).

intime,interactionstepsandeffortwerecomparedforthefirstandsecondit-eration.Learningeffects(LE)andeffectsbasedontheassistance(AE)couldbeseparatedbymeansofthelogfiledata(seetable3).Aclearreductionoftempo-raleffortcouldbeobservedfortheseconditerationforallconditionseventhoughthiswasnotsignificantforthePHgroup.InterestinglyanincreaseininteractionstepswasobservedforthePHcondition,thiscanbeduetotheadditionaleffortintabbingthroughthemorecomplexlistassistantinterface.AssistanceeffectwashighforPHbutcorrelationcouldnotreachsignificance.Interactionstepscouldalsobereducedconsiderably,wheretheeffectofassistancecontributedsignificantlytotheeffectonlyforthephysicallydisabled(T-Test:p=.04).Men-talandmotoreffortwaslowforallconditions(seefigure7),usersoftheND2conditionshowedasignificantdecreaseofphysicaleffort(T-Test:p=.041),whileuserswithoutmobiledevice(ND1)showedlessmentalloadthanthephysicallyhandicapped.Whileeffortwasdecreasingfortheuserswithmobileassistantsratingsofuserswithoutmobiledeviceindicatedincreasingeffort.

Question3:Acceptancehasbeenassessedbymeansofanine-scaleusabilityquestionnairedesignedforthisstudy.Theaverageratings,thatareshowninfigure8,areverypositive.8.3

DiscussionoftheResults

TheresultsoftheevaluationstudyshowedanencouragingresultfortheEM-BASSIconcept,confirmingthefeasibilityandeffectivenessalsoastheacceptanceofmobileassistancefordisabledandnon-disabledusers.Inthefollowingwewillgothroughthethreequestionsraisedabove.

Answer1:Physicallyhandicappeduserswereabletocompletethetasksequallywellorevenbetterthannon-disableduserswithoutmobiledevice.Thisisespeciallyencouragingasthisusergrouppreviouslywasexcludedfromtheuseofsuchsystemsatall.

Fig.6.a)Ratingofsubjectiveeffortfornon-disableduserswithoutmobiledevice(ND1),non-disableduserswithmobiledevice(ND2),andphysicallyhandicappedwithmobiledevice(PH)duringthestudy.b)Ratingontheninescalesoftheusabilityoftheusabilityquestionnaireforthetwousergroupsandmobiledevices.

Answer2:Usersdidnotprofitclearlyfromtheuseofmobiledevicescon-cerningeffortandtimesaving,alsosubjectivetaskloadwashigher.Howeverassistancethroughpersonalizationledtocleardecreaseininteractioneffortforthenon-handicappeduserswhilephysicallyhandicappedneededmoreinterac-tionsteps.Atthesametimebothgroupsshowedacleartimesavingeffectduetotheshoppingassistance.Usersofmobiledevicesexhibitedsteeplearningcurveswhichcouldbecomemorerelevantiftheygetusedtosuchdevices.Theten-dencyofanoveralldecreaseofeffortfortheusersofthemobiledevicesagainstanincreaseforuserswithoutmobiledevicesindicatesthereducedeffortcausedbytheadaptedinteraction.

Answer3:Users’acceptanceofthemobiledeviceswasclearlypositive.Es-peciallythedeviceforphysicallyhandicappedpersonswasperceivedasagoodsupport,errortolerantandfun.

Allinallthesystemwasperceivedasaclearsupportbytheusers,nomatterifwithorwithouthandicaps.Inthecaseofthephysicallyhandicappedhowever,thisoffersanewwaytoaccesssuchserviceswithoutrequiringtomuchaddi-tionalmentalorphysicaleffort.Futureworkwilladdresstheimprovementoftheinteractiontechniquesandthereductionofinteractioneffortwiththeassistance.Furtherrefinementofinterfaceadaptationwillsurelyreduceinteractioneffortbeyondwhathasbeenshowninthisstudy.

9RelatedWork

Thechallengetoprovideuserswithanalternativeinterfacetodevicesandappli-anceshasbeenaddressedinanumberofotherprojectsrecently.TheArchimedesProject[2]offersanalternativeaccesstoinformationtechnologyoveranabstractinteractionprotocolwhichallowstoconnectso-called“accessors”implementing

specialI/Otechnologiestocomputersofdifferentplatforms.ThePebblesprojecthasdevelopedaplatformwheremobiledevicescanbeusedtocontrolaPCormediaappliances[20]throughaautomaticallygenerateduserinterfacebasedonXML[21]andcombiningthiswithenhancedassistivetechnology.Thefocusliesonahomescenariowhereasingleuserinteractswithpersonalappliances.[21]haveshownthatthiscanbeusedtosupportusersininteractionathome,butdy-namicdiscovery,sessionmanagementandotherfeaturesnecessaryforthepublicdomainarenotavailable.AlsotheV2StandardaddressesasimilarscopewiththeAlternativeInterfaceAccessProtocol(AIAP),whichhassomeofitsrootsintheUniversalRemoteConsole(URC)fromTraceCenter[14,22,36].WhiletheinterfacedescriptionlanguageemployedbyV2isbasedontheXFormsstandardandemploysXPathforconstraintsandrelations,asthisisthecasewithXUIPebblesdefinesitsproprietaryformat.EventhoughexhibitingthepotentialofmultimodalpresentationthishasnotyetexaminedneitherbyPebblesnorbyV2whichisstillinadraftstate.Nichols[22]statesthatinputsynchronizationmightstillbeanopenissueinPebbles.Theproblemofabstractspecificationofstyleinformationalsoisnotaddressedwhiletheconceptofhigherlevelwidgets(e.g.mediacontrol)seemspromising,especiallyinthelightofdesignpatternbaseddevelopment[37].Toolkits,liketheJavaAccessibilityAPIorMircorosftActiveAccessibilityareinmostcasesrestrictedintermsofvisualuserinterfacesortheannotationofdata.Theyareoptionaltouseandthereforeoftennotconsideredneitherduringdesignnorduringimplementation.Accessiblesystemsrequireamuchdeeperintegrationofadaptationandconfigurationmechanismsthanthisistruetoday.

10SummaryandOutlook

Whatdowehaveachievedtillnow?Weprovidedalternativeaccesspossibilitiestoterminalsandtheirservicesbymeansofpersonalmobiledevicesthatareadaptedtothespecialneedsofdisabledpersons.Throughanamodalgenericinterfacedescriptionlanguagetheterminaltransmitsitsuserinterfacetothemobiledevices,whichprovideadaptedandoptimizedinteractiondevicesandpersonalizedandcontextawareassistance.ThesystemisbasedonthelayeredEMBASSIinteractionarchitecture.Fourdifferentpersonalmobiledeviceswereintroduced,onefornon-disabled,oneforvisuallyimpairedpersons,oneforphys-icallyhandicappedpersons,and,oneforelderlypeople.Eachdevicerealizesan-otherconfigurationofinputandoutputdevicesinordertoprovidemultimodalinteractionsupportsuitabletospecialusergroups.BymeansofthegenericEMBASSIarchitectureotherdevicescanbedevelopedusingthesamesoftwarebutotherinputandoutputcomponents,addressingotherusergroups.Afterthepresentationofthetechnicalbackgroundofoursystem,theresultsofthefinalevaluationstudywaspresented.Theresultsoftheevaluationindicatesthattheapproachofprovidingindividualandadaptablemultimodalaccesstopublicinformationsystemswasaccepted,efficientandeffective.Especiallythoseusers

whichinotherwayswouldbeexcludedfromtheuseofpublicterminalsclearlyprofitfromsuchasystem.

Progressiveminiaturizationandincreasingavailabilityofpowerfulmobilecomputingdevices,suchasPDAs,Smartphones,andalike,willpushforwardacceptanceofsuchsystemsbyusersandserviceproviders.Miniaturizationofassistivetechnologywillleadtoabroaderrangeofsuchsystemsavailableformobileusewheretoday’ssystemsaremainlydesignedforstationaryuse.WiththeEMBASSIapproachwethinkthataexperimentalplatformforsuchserviceshasbeendevelopedwhichcanbeextendedforfurtherapplicationsandrefine-ments.Themultimodalapproachtowardsuniversalaccessibilityinthepublicdomainhasproventobefeasibleandwellacceptedbytheuser.

11Acknowledgements

ThisworkwaspartiallyfundedbytheGerman“BundesministeriumfrBildungundForschung,BMB+F”throughtheFocusProjectEMBASSI(BMB+F-No.FKZ01IL904U8).WewouldliketothankallEMBASSIprojectpartnersfortheirhelpfulideasandsupport.EspeciallywewanttothankMrs.MaritaEngefromtheHumboldtUniversityofBerlinforsupportingtheevaluationsandthefollowingcompanies:IngenieurbroDr.Seveke,Dresden,ForschungsinstitutTech-nikfrBehindertederStiftungVollmarstein,Wetter/Ruhr,APGGmbH,Hamm,andHandyTechGmbH,Horbfortheirgreatco-operationduringthedevelop-mentofthemobiledevicesandtheirmultimodalfeatures.Butthegreatestthankandcommendationwehavetogivetoallparticipantsoftheevaluationstudiesfortheirsystemtrails,theirworkduringtheevaluationsandtheirindispensablesuggestions.

References

1.Abrams,M.,Phanouriou,C.:UIML:AnXMLlanguageforbuildingdevice-independentuserinterfaces,ProceedingsoftheXML’99,Philadelphia,USA,19992.TheArchimedesProject,http://archimedes.stanford.edu/,StanfordUniversity,USA,2002

3.Clemens,D.etal.:IndividuallyAssistedTextEntrywithSituationalandContextualPrediction,Proc.the8thInternationalConferenceICCHP2002,Linz,Austria,20024.Czaja,S.J.,Lee,C.C.:DesigningComputerSystemsforOlderAdults,in:TheHuman-ComputerInteractionHandbook,LawrenceErlbaumAssociates,USA,p.413-425,2003

5.DubinkoM.etal.:XForms1.0,W3CRecommendation,October,2003

6.EisensteinJ.,Vanderdonckt,J.,Angel,P.:Applyingmodel-basedtechniquestothedevelopmentofUIsformobilecomputers,Proc.oftheConf.onIntelligentUserInterfaces(IUI’01),NewMexico,USA,2001

7.Enge,M.,Massow,S.:Needsforassistanceofvisuallyandphysicallydisabledandnon-disabledpersonswhenusingmnoney-/cashdispenders,in:deWaard,D.etal.(eds):Human-ComputerInteraction:Education,ResearchandApplicationinthe21stCentury,ShakerVerlage,Maastricht,TheNetherlands,p.263-266,2002

8.Forkl,Y.,Hellenschmidt,M.:MasteringAgentCommunicationinEMBASSIontheBasisofaFormalOntology,ProceedingsoftheISCATutorialandResearchWork-shop,MultimodalDialogueinMobileEnvironments,KlosterIrsee,Germany,20029.GeneralServicesAdministration,Section508:Theroadtoaccessibility,2002

10.Grundy,J.,Zou,W.:Anarchitectureforbuildingmulti-devicethin-clientwebuserinterfaces,Proc.ofthe14thConferenceonAdvancedInformationSystemsEngineer-ing,Toronto,Canada,2002

11.Hart,S.G.,Staveland,L.E.:DevelopmentofaNASA-TLX(TaskLoadIndex):ResultsofEmpiricalandTheoreticalResearch,in:Hancock,P.A.,Meshkati,N.(eds):HumanMentalWorkload,Elsevier,Amsterdam,TheNetherlands,p.139-183,1988

12.Hellenschmidt,M.,Kirste,T.,Rieger,T.:Anagent-basedApproachtoDistrib-utedUserProfileManagementwithinaMulti-ModalFramework,ProceedingsoftheIMC2003,Rostock,Germany,2003

13.Herfet,T.,Kirste,T.,Schnaider,M.:MultimodalAssistenceforInfotainmentandServiceInfrastructures,Computers&Graphics,25(4),p.581-592,2004

14.INCITS,DraftV2WorkingDocument:ArchitectureoftheUniversalRemoteCon-soleSpecification(AIAP-URC)oftheAlternateInterfaceAccessProtocol(AIAP),http://www.incits.org/tchome/v2htm/docs/V2/02-0086/v2020086.htm,2002

15.ISO9241,Ergonomicrequirementsforofficeworkwithvisualdisplayterminals(VDTs),InternationalStandardsOrganization,2001

16.ISO13407,Usercentereddesignprocessforinteractivesystems,InternationalOr-ganizationforStandardization,1999

17.Kirste,T.,Rapp,S.:ArchitectureforMultimodalInteractiveAssistantSystems,Proceedingsofthestatusconference“Mensch-TechnikInteraktion”,Saarbr¨ucken,Ger-many,2001

18.Labrou,Y.,Finin,T.:AProposalforanewKQMLSpecification,ComputerScienceandElectricalEngineeringDepartment,UMBC,TRCS-97-03,Baltimore,MD,USA,1997

19.Martin,D.L.,Cheyer,A.J.,Moran,D.B.:TheOpenAgentArchitecture:AFrameworkforBuildingDistributedSoftwareSystems,AppliedArtificialIntelligence,p.91-128,1999

20.Myers,B.A.:UsingHand-HeldDevicesandPCsTogether,in:CommunicationsoftheACM,44(11),p.34-41,2001

21.Nichols,J.,etal.:GeneratingRemoteControlInterfacesforComplexAppliances,Proc.oftheACMSymposiumonUserInterfaceSoftwareandTechnology(UIST’02),Paris,France,2002

22.Nichols,J.,Myers,B.A.:ReportontheINCITS/V2AIAP-URCStandard,CarnegieMellonUniversity,Pittsburgh,USA,2004

23.Norman,D.A.:ThePsychologyofEverydayThings,BasicBooks,USA,198824.Newell,A.F.,Carmichael,A.,Gregor,P.,Alm,N.:InformationtechnologyforCog-nitiveSupport,in:TheHuman-ComputerInteractionHandbook,LawrenceErlbaumAssociates,USA,p.4-480,2003

25.Oviatt,S.:MultimodalInterfaces,in:TheHuman-ComputerInteractionHandbook,LawrenceErlbaumAssociates,USA,p.286-301,2003

26.Richter,K.:AnadvancedconceptofassistanceformobilecustomersofPOS,Proc.ofE-BusinessandE-Works,Prague,CzechRepublic,2002

27.Richter,K.:GenericinterfacedescriptionsusingXML,Proc.ofComputerAidedDesignofUserInterfaces(CADUI’02),Valenciennes,Frances,2002

28.RichterK.,Enge,M.:Multi-modalframeworktosupportuserswithspecialneedsininteractionwithpublicinformationsystems,Proc.ofHuman-ComputerInteractionwithMobileDevicesandServices(MobileHCI2003),Udine,Italy,2003

29.Sears,A.,Young,M.:PhysicalDisabilitiesandComputingTechnologies:AnAnaly-sisofImpairments,in:TheHuman-ComputerInteractionHandbook,LawrenceErl-baumAssociates,USA,p.482-501,2003

30.Shneiderman,B.:Designingtheuserinterface:strategiesforeffectivehuman-computer-interaction,3ndedition,Addison-Wesley,1998

31.Stephanidis,C.,Akoumianakis,D.,Sfyrakis,M.,Paramythis,A.:Universalacces-sibilityinHCI:Process-orienteddesignguidelinesandtoolrequirements,Proceedingsof4thERCIMWorkshoponUserInterfacesforall,Stockholm,Schweden,199832.Stephanidis,C.:DesigningforallintheInformationSociety:Challengestowardsuniversalaccessintheinformationage,Report,ERCIMICST,1999

33.Stephanidis,C.:UserInterfacesforall:Concepts,Methods,andTools,in:HumanFactorsandErgonomicsSeries,NewJersey,USA,2001

34.TheEuropeanCommission:Europeanyearofpeoplewithdisabilities,200335.WorldWideWebConsortium(W3C),WebAccessibilityInitiative(WAI),200436.Vanderheiden,G.:Interactionfordiverseusers,in:TheHuman-ComputerInter-actionHandbook,LawrenceErlbaumAssociates,USA,p.397-400,2003

37.vanWelie,M.,vanderVeer,G.C.:PatternLanguagesinInteractionDesign:StructureandOrganization,in:Interact2003,2003

38.XMLUserInterfaceLanguage(XUL),http://www.mozilla.org/projects/xul/,Au-gust,2004