Automated discovery of search interfaces on the web

JaredCope

AustralianNationalUniversity

Jared.Cope@accc.gov.au

NickCraswellandDavidHawking

CSIROMathematicalandInformationSciencesnick.craswell@csiro.au,david.hawking@csiro.au

Abstract

Websearchenginesworkwellforfindingcrawlablepages,butnotforfindingdatasetshiddenbehindWebsearchforms.Wedescribeanoveltechniquefordetectingsearchforms,whichcouldbethebasisforanext-generationdistributedsearchap-plication.WeuseautomaticfeaturegenerationtodescribecandidateformsandC4.5decisiontreestoclassifythem.Intwotestbeds,wegetanaccuracyofmorethan85%andapre-cisionofmorethan87%.Oneofourdecisiontreesiseffectiveonbothtestbeds,suggestingthatitisausefulgeneral-purposetree.

Keywords:retrieval,machineWorldWidelearningWeb,distributedinformation1

Introduction

Manycannotusefultechnology.besearcheddatasetsusingontheWebareuncrawlable,soinformationHowever,andretrieval,usingconventionalsearchenginesuchtechniquesdatasetsofdistributednaryForaccessedexample,automatically.

canbelocatedtoincludetheOxfordEnglishhave(pagestohttp://www.oed.com/crawlit:a),asearchengineDictio-wouldOEDisandasubscriptionsiteandisfollowingprocessuncrawlabletheirofhyperlinks.recursivelyHowever,downloadingtheservice,soforanumberofreasons.Itareenduserswouldneedbothsubscriptions.theengine’sItscrawlerpagesaccessiblenotreachableform).tocrawlersbyfollowing(usersaccesshyperlinks,pagessoarein-itsexclusionmainFinally,contenttheareas,siteforbidsviatheenginesstandardfromviaforcrawlingasearchnon-excluded,Evenif(robots.txtthepages).

robotswerepublic,hyperlinkedandsivelythereareproblemswithcomprehen-words,crawlingand2436600suchasite.Ithasmorethan6151on219800etymologies.quotations,Ifeach139900ofthesepronunciationsappearedworkoneOEDtrafficpage,andacrawlserverwouldgeneratenoticeablenet-callysiteeachdefinitionload.appearsHowever,onmanyinthedynami-currentquotations,generatedpages(alone,withetymologies,withgieswithpronunciations,withbothetymolo-thenationsiteandwouldquotations,requireandsophisticatedsoon).Efficientlyduplicatecrawlingnotsearchbemethods.betterthanEventhen,thecrawledsearchmightelimi-cludesisintegratedthewithoriginaltherestOEDofsite’s.TheOEDhardto“advancedemulateinsearch”acrawledcapabilitiesthesystem.

whichsitewouldandin-beCopyright󰀁

c2003,AustralianComputerSociety,Inc.Thispa-perappearedatFourteenthAustralasianDatabaseConference(ADC2003),Adelaide,Australia.ConferencesinResearchandPracticeinInformationTechnology,Vol.17.XiaofangZhouandKlaus-DieterSchewe,Ed.Reproductionforacademic,not-forprofitpurposespermittedprovidedthistextisincluded.

1DiscoveryMeta searcher???????2Characterisation3SelectionMeta searcherUserqueryMeta searcherenginessearch enginessearch 4Query translation5Result mergingUserMeta searcherresultsUserMeta searcherenginessearch enginessearch Figureretrieval.

1:FiveproblemsofdistributedinformationhaveThetrendontheWebisformoreWebsitestotems,databasecontentdynamicbackcontent,ends,multiplecontentviewsmanagementofsys-makesanditandmoreaccessdifficultrestrictions.Eachofthesethefactorssametant.

makeslocalsearchenginestodoarelativelycomprehensivemoreimpor-crawl,useMethodsvidewhateverforsearchdistributedinterfacesinformationareavailableretrievaltocanthanasearchcrawling,unifiedsearchtheyassistexperienceusersbyforselectingtheuser.theRatherpro-rightresultsinterfaces,queryingthemandpresentingtheirdistributedThisinpaperanintegratedconsiderslist.

themostbasicproblemofWeb.trievalItconsidersinformationhowretrieval,adistributedintheinformationcontextoftheoverthewhichsystemitwillcanidentifytheWebsearchsystemsre-ers.OEDsearchinterfaceoperate,andwithmanycandidatesthousandsincludingofoth-2

Motivationtrieval

—Distributedinformationre-Mostconsideredworkindistributedinformationretrievallection,queryfourtranslationmainproblems:andresultcharacterization,hasmerging.Before

se-itavailableisqueried,lectssearchtheinterfaces.metasearchThensystemgivencharacterizesaquery,itthese-andasubsetofusefulsearchinterfaces,queriesthemprecedesThispresentspapertheirresultstotheuser.

(Figurecover1).theTheotherconsidersmetafour,anoverlookedproblemthatsearchdiscoveryofsearchinterfacessuchsteps.

aaset,setbeforeofsearchitcaninterfacessystemproceedorwithbemustprovidedfirsttheotherwithdis-fourPerkowitz,Verylittleworkhasbeendoneinthisarea.investigatedinteractAnwiththeDoorenbos,informationproblemofEtzioni&Weld(1997)resourcesautomaticallyonthelearningtoforms,agentwasdevelopedforfindingshoppingInternet.searchwhichbasedonsimpleheuristicsfordiscardingthosepercatedalsoareclearlysomethingelse.Thecurrentpa-iedapproach.classifiesforms,butwithamoresophisti-sionselectionInoffutureratherDreilingerworkthan&Howe(1997)stud-foreshadowsdiscovery,buttheirdiscus-autoparticular,couldpilotroamingtheythesuggestWebthatforanouragentcurrentactingwork.ontheametanewlyleaddiscoveredtoanewresourcessearchparadigm,newsearchresourcesareincorporatedespeciallyifmetaOncesearcher.

intoissearcherasearchmayinterfacefindouthaswhatbeendiscovered,theproblemavailableisillustratedfromtheinterface.inbox2ofThissortofinformationfigurecharacterization(1997)Gravano,Chang,Garc´ıa-Molina1.&Paepckesearchersproposedastandard,STARTS,toaidmetaprotocolinchoosingthebestsourcestoquery.Thetiontheaboutrequiresthemselvessearchinterfacestoexportinforma-Howeverlistofinastandardformatsuchasoninwordspractice,containedinthedocumentsindexed.thattheprotocol.allWebsearchbecausetheresuchisanoprotocolauthorityhastolimitationsmakesurepliantqueryengines.PresentlyinterfacesCallantherecooperateandimplementthe&areveryfewSTARTScom-interfaces,basedsampling,whichConnellinvolves(2001)queryingpresentedbuildingments.adownloadingsomeresultsdocumentssearchandareevenmoreResourceresourcewidelyavailable,descriptionsdescriptionbasedonthosedocu-becausebuiltfromprobequeriesingMehranwithifsearchthemetainterfacessearcher.areIpeirotis,notexplicitlytheyareavailableGravanocooperat-tion(2001)lookedatautomatingtheclassifica-&hierarchy.ofsearchinterfacestobuildaYahoo!-liketopicqueries,cation.

toTheirmaximizesystemthelikelihoodusescarefullyofcorrectchosenclassifi-probetionForreasonsofalwaysandsearchappropriatepossiblyefficiency,improvingreducedtosendallresultquality,networkitconges-isnotaquery.subsetengines.oftheSelectionsearchenginesisthequeriesproblemtoofallselectingknownurefollowing1.ManualThisproblemstudiesselectionisillustratedforainparticularbox3ofuserFig-considerisautomaticpossiblebuttrivial.ThewellCallan,Lu&Croft(1995)selection.

introducedthetermknownSTARTSoccurrenceCORIserverselectionfunction,basedonareorprobeinformationqueries.whichisavailableviaandGlOSSleyCVV(Gravano,(Yuwono&Garcia-MolinaOthersimilarLee1997).&Craswell,Tomasicfunctions1999)Bai-and&probeCVVHawking(2000)evaluatedCORI,vGIOSStionqueriesusingandresourcefounddescriptionsgeneratedfromenvironments,algorithm.DreilingerincludeOther(Hawkingapproaches,CORIwasthebestselec-&Thistlewaiteforuseinvaried&Savoy&Howe1997,Fuhr1999,Rasolofo,Abbaci1999,Query2001).

translationistheproblemoftransforming

therecipientuserqueryinsearchintoengine.alanguageThisthatproblemisacceptedisillustratedbythedifferentbox4ofFigure1.Translationissuesariseguages,(Chang,forsearchenginesacceptdifferentquerywhenlan-&keywordsBorghoffGarc´exampleis1998).ıa-MolinawhenausefulHowever,&processingPaepckebooleanquerieslowestcommononthe1996,WebChidlovskiiastringofcombinedResultmergingistheproblemofdenominator.presentingtheexample,umentsinresultsarankedofengineslistwithintheausefulmostrelevantfashion.doc-Fortratedrankednearthetop.Thisproblemisillus-cols.Againinboxitis5ofFigure1.

formlyWith(Gravanoacrosscooperation,possibletorelyoncooperativeproto-allsearchengines,rankingcanbedoneuni-generateilarcomparableetal.1997)matchorsimplyforexampleSTARTSscores.useHowever,algorithmsforwhichsim-areranksEffectivenotreasonswidelyofnoncompliance,cooperativeprotocolsmergingusedoncanthebeWeb.

ontiontheassigneddownloadedbycontentstheselectedbasedonthescoresorofthesearchinterfaces,or&(Lawrence&Giles1998).Craswell,documentsinques-loadThistlewaiterankingmethod(1999)foundthedocumentHawkingdown-referencewithtosetofahighbemosteffective,particularlywhencollection-widequalityrankingalgorithmandatrieval,Withdoneitallisthissurprisingworkinthatdistributedstatistics.

solittleinformationworkhasre-otheronworkinterfacereliesondiscovery,havingaparticularlysetofknownsinceinterfaces.allbeenthe3

InterfaceDetection

Aofsearchquery,itemswithoutinterfacealteringallowsathem.usertoThesearchsomesetitem(s)bytooftypinginterest.orselectingResultsmightoptions,userentersabetoadescribetheingitemsfeelingitems(usual(aphonesearchlistenginesearch)results),orapagepagecontain-linkingsomehowlucky”inGoogle).Theasinglepage(“I’mareThevastmatchitem(s)foundshouldmajoritythequery.

typesHTMLlargeofinterfacesforms.ofsearchinterfacesontheWebsuchForasthisJavareasonGUIs.weItisignoreeasytootherfindandforescanningnumbersofHTMLforms,bycrawlingtheWebform,thetargetisitdetectioncrawledpagesforformtags.There-aproblembecomes:givenanHTMLin(fortermsURLsearchinterface?Note,mostformshaveaofforms(action),soalthoughourclassificationissearchWebexamplesearchTablewesometimesrefertoformtargetsinterfaces3andTablecommonly10).

allowuserseralographiccrawl“itemorsets”fromsuchsingleas:site,Webproductspagesfromtoforagen-bibliographiclocations,people,dictionarydefinitionssale,ge-orterfaceslistincludeentries.Formswhicharenotsearchin-shop,subscriptiondiscussionshipWeb-basedforms,emailformspurchasegroupandformsinterfaces,mailingWebsiteinanonlineformsforms.InourANUtestset,aboutmember-facesWhenwere50%ofHTMLclassifyingsearchinterfaces.

HTMLpre-queryornon-searchapproach,orapost-queryinterfaces,formsapproach.itispossibleintosearchtotakeinter-acanInbeanalyzed,theforminitselforderandtothepageIncontainingapre-queryitbepost-querypagessenttotheclassification,systeminquestion,onemakeortheandmoreclassification.queriescanapproachusedsendforasreasonsanindicator.ofpoliteness:Wechoosetheresultingitistheimpolitepre-querysiongrouparbitrarysimplyqueriesforthetoasakepurchaseofclassification.

formoradiscus-toFormparametersforautomaticfeaturesnamevalueparameterparameterforforaninputcontroldistinctnamewordparameteranfromaforinputformaformcontrol

action

Tableformfrom1:Thiswhichtablewedescribesgeneratefourfeatures.

placesinaHTMLANUset

RandomWebSet

Numfeatures

597

861

Tablematically2:Numberofsets.

forboththedistinctANUandfeaturesrandomgeneratedWebtrainingauto-weThechineuseforremaindersearchinterfaceofthissectiondetection.describesWetakemethodsturesclassificationuponlearningwhichapproach,sowefirstdescribetheama-fea-methodwebaseitself.ourclassification,thenthe3.1

Featuregeneration

HTMLexploitedformscontaintiontoobtainacomplexrichsetofstructurefeatures.thatThiscanbeatingdescribesobtainingfeaturesoneforanmethodHTMLforformautomaticallysec-withgener-tion.

arepresentationusefulforinterfacethegoaldetec-ofofFeaturescanautomaticallyinformsbasedonvaluesforcertainbegeneratedparametersforfoundaseteredtheinHTMLthispaperformaremarkup.intableTheparametersconsid-stringTheinofdistinctcharacterswordthatfromaform1.

actionreferstoaHTTPtheformaction(ignoringappearthecolonbetweenrequiredslashesbythe(/)isprotocol).Forexample,iftheactionforaformdistincthttp://search.anu.edu.au/external/thentheexternalwordswouldbehttp,search.anu.edu.auandbasedFeatures.

canalsobeautomaticallyHTMLonandform,thefortypesexampleofformcontrolspresentgeneratedinthegeneratedpasswordahavingformhavingfromcontrols.Inexistenceaddition,offeaturestextcontrolscanbeathesinglenumbertextofcontrolcontrols,versusfortheexampleingToplefeaturesillustratemultiplethetextprocesscontrols.

formofautomaticallygenerat-forinfigurefrom2.ThethetopHTMLboxshowscode,considertheexam-thataformandthebottomboxthetheresultingHTMLmarkupfeaturescarriedInarethisautomaticallyandoutstudy,separatelyautomatedgenerated.

onbothfeaturegenerationwasationThetheresultingrandomWebtheANUtrainingsetfeaturestrainingfromset.

Instead,methodtomaticallyTableare2summarizestoonumerousthistoautomatedlistindividually.gener-trainingsets.

generatedfeaturestheobtainedtotalnumberfromtheoftwoau-theTablecomparedstructure2suggestsandcontentthatthereofformsismorevariationinnumberberthethoseofwiththeANUWebdomain.ontheAlthoughWebthantheinformstheinANUtherandomsamplebyWeb18%sampleoutnum-domnumber44%.WebsampleofuniqueoutnumberfeaturesthefoundANUfrom(260samplethevs219),ran-byphenomenon.

Section7presentssomeconsequencesofthissearchnon-searchANUtrainingset149(t:34)70(t:43)ANUtestset

185(t:24)199(t:60)RandomWebtrainingset150(t:80)110(t:88)RandomWebtestset

150

(t:81)

113(t:92)

Tableber3:Trainingandtestsets.Listedaretheeachoftargetedset.formsbyAlsoofthelistedeachtype(searchandnon-search)num-informsare(t:).

thenumberofuniqueURLs3.2

ClassificationHavingapplyingautomaticallygeneratedInourthisaclassificationalgorithmarichisasetsimpleoffeatures,matter.2001).

maincasealgorithm,wechooseastheimplementedC4.5learninginalgorithmWeka(UWasimplementedWechosethetypeinmultiplealgorithmbecauseitiswellknown,portantly,offeaturesgeneratedplaces(mostlyandbinary).amenableMoretotheim-(tree)thealgorithmproducesaclassificationruleitsnestedentiretywhichiseasilyunderstandable,publishableinthatconditionals.andimplementedFurther,testsinanyinSectionlanguage7.2usingshowtive.otherclassifiersarenotsignificantlymoreeffec-4

Testbeds

CollectionsanfromtwodomainsdomANUareusedinthisstudy,byWeb(AustralianNationalUniversity)andran-searchtheauthorsset.Collectionsandexampleswerelabelledmanuallyassampledeitheralectionsinterfaceornonsearchinterface.Thesecol-withprovideatestinggroundforexperimentingficationHTMLsuccessformoffeaturesanyfeaturesanddeveloped.evaluatingtheclassi-4.1

ANUcollection

AANUtraining20016crawlWebsetofpageswasobtainedfromhostsintheofdomainaround(anu.edu.au430000pages.),usingWeaFebruaryin500identifiedURLpagesorder.containingHTMLformsandidentifiedsortedthemtheofatrainingThen,setselectingof200pages.every30thForsimplicitypage,wemultiplejudgingforms,andlabelling,givingawetrainingthensplitfilescontainingandThisoreachsetwasthenmanuallyjudgedsetofby219theforms.authorsthereanonsearchformwasinterface.labelledAsaseitherasearchinterfaceinterfaceswere149searchinterfacesaresultandof70thisnonlabelling,searchcrawl.Atestsetinthisofformstrainingwasalsoset.

obtainedfromthethepickedformsThewassameapplied,procedureexceptinchoosingsamethatandjudgingwithsureandthataatdifferentrandomfromthecrawl(three300applicationsformswerethetestoffsetsetisinthelistof6500,tomakemultiplechoosingdifferenttothetrainingset,judgingandthisformsevery60thpage).Afterdecomposingtestfromset,atherepagewereinto185theirownfilesanditThe199testnonsetsearchislargerinterfaces.

searchinterfacesthanthetrainingprocedurewasgatheredwasmorelaterrefined.

whenthejudgingandsetlabellingbecauseaction=”http://www.altavista.yellowpages.com.au/cgi-bin/query”>

inputType-SingleTextinputType-SubmitinputType-Hidden

inputType-text:Name=q

inputType-submit:Name=submit2inputType-hidden:Name=mssinputType-hidden:Name=pginputType-hidden:Name=whatinputType-hidden:Name=encinputType-hidden:Name=klinputType-hidden:Name=localeinputType-submit:Value=searchinputType-hidden:Value=simpleinputType-hidden:Value=qinputType-hidden:Value=webinputType-hidden:Value=iso88591inputType-hidden:Value=xxinputType-hidden:Value=xxFormName:altavistaactionWord:http:

actionWord:www.altavista.yellowpages.com.auactionWord:cgi-binactionWord:query

Figure2:Thisfigureshowsautomaticfeaturegenerationinaction.ThetopboxcontainssomesampleHTMLcodeforthedeclarationofaform.ThebottomboxshowstheresultingfeaturesderivedfromtheHTMLformcontent.

Textarea controlNYSingle Text controlNONNYSEARCHSubmit controlvalue = ‘search’SEARCHNYvalue = ‘submit_query’Submit controlSEARCHNYNONSEARCHSEARCHFigureset.The3:treeDecisionwasbuilttreewithbuilt597fromavailabletheANUfeatures.

training4.2RandomWebcollection

AnotablysamplesetofpageswasobtainedfromtheWebandofabovetheWeboutsidetheANUdomain.Sinceafullcrawltestsetwaswasofsearchneedednotinterfacesinavailable,ordertoadifferentobtainastrategytrainingfromandcom/ThetorandomobtainisWebandnon-searchinterfaces.adirectorysitehttp://www.searchengineguide.asampleofsetsearchinterfacesandwasusedtedmanuallyWebcollection.1bysiteowners,Theseofsearchratherinterfacesinterfacesthanarefortheautomati-submit-callyadetectedbusiness,broadrange.samplesocietyofSearchtopicsinterfaceswerechosenfromandscience.thatindexFromnewsandmedia,randomof150searchinterfaceswereselectedthissource,foratheberandomWebtrainingsetand150wereselectedtheforaWeb.veryrandombroadbecauseWebtestthisset.WebThissitemethodactsasisapointerarguedtotoonealloverspecificSoalthoughrangesite,thetheofactualsearchinterfacesontheinterfacesinterfacesactuallywereobtainedoriginatefromsitesTolinkedobtaintheWeb.

fromfromthesethttp://www.searchengineguide.ofnonsearchforms,alistofWebcom/theirlyzedhome-pages.andhttp://www.dmoz.org2judgedtofindcandidateTheforms.homeThesepageswerefollowedtoformswerethenwereana-thenwereWithkeptandiftheywerenonsearchinterfaces,theywerethistomakeupthesetofnonsearchinterfaces.theeventuallymethod,foundasetforofthe110trainingnonsearchsetandinterfaces113forbothTabletestset.

theANU3comparesandrandomthetrainingWebcollections.andtestsetsfor5

Results

DecisiongorithmtreesarebuiltwiththeC4.5learningal-fromusingtheautomaticallygeneratedfeatureswasobtainedsection3.1.fromThe(UWimplementation2001).

ofC4.5usedpredictedsearch

predictednonsearchactualactualsearchnonsearch

1462

368

Tableusingrules4:ConfusionderivedfrommatrixFigureforthe3.

ANUtrainingsetpredictedsearch

predictednonsearchactualactualsearchnonsearch

1809

5190

Tablerulesderived5:ConfusionfromFigurematrix3.

fortheANUtestsetusing5.1DecisiontreefortheANU

ThiseratedsectiongeneratedfromthepresentsANUtrainingthedecisionsetusingtreethatautomaticallywasgen-theFigurefeatures.

3showstheoferatedtheANUclassificationtrainingsetiswithdecisiongiven597treeconstructedforinfeatures.table4.ThesuccessprecisionfromthetreehaveanaccuracyofRules98%gen-rulesTableof5shows99%.

andatheclassificationsuccessforthisareclassificationappliedontheis96%ANUandtesttheset.precisionThewhenaccuracytheis95%.5.2

DecisiontreefortherandomWeb

ThistheeratedrandomsectionWebpresentstrainingadecisionsetusingtreeautomaticallygeneratedfromgen-theFigurefeatures.

4showsthedecisiontreeconstructedfeatures.randomtionTableWebsetwith861automaticallygeneratedforUsingonclassificationthetherulestraining6showstheresultsofthisclassifica-fromsetwiththetreefromFigure4.of96%.

hasanaccuracythedecisionof92%treeandinfigureaprecision4,thetheTabletestset7showswiththetherulesresultsgeneratedofthisfromclassificationthedecision

on1

personalcorrespondenceon25/09/01withRobertClough,Web-master2

ofSearchEngineGuide.AdirectorywhichactsasapointertoabroadrangeofWebdocuments.

predictedsearch

predictednonsearchactualactualsearchnonsearch

1355

15105

Tableingset6:usingConfusionrulesderivedmatrixforfromthetherandomtreeinWebFiguretrain-4

predictedsearch

predictednonsearchactualactualsearchnonsearch

13120

1993

Tablesetusing7:ConfusionrulesderivedmatrixfromforthethetreerandominFigureWeb4

testSubmit controlvalue = ‘search’NPassword controlNaction word‘search’NONSEARCHYYSEARCHNText controlname = ‘email’YNYSEARCHTextarea controlNYNONSEARCHaction word‘cgi’NYNONSEARCHHidden controlSEARCHNYSubmit controlImage controlNYNYMultiple TextcontrolNONSEARCHSEARCHSingle TextcontrolYNYNNONSEARCHSEARCHNONSEARCHSEARCHFigure4:DecisiontreebuiltfromtherandomWebtrainingsetwith861availablefeatures.

Rules generated from automatic featuresANU Domain71% / 73%Training 95% / 91%SetTestSetRandom RandomWebWebTrainingSetTestSet93% / 92%76% / 75%Figurerepresent5:(dashedtraining/testCrossvalidationacrossdomains.Circlesrule).forthe“random”setsruleandandlinessolidrepresentfortestsages,accuracyEachlineandisassociatedprecisionrespectively.

withapairofthepercent-ANUtree85%inandfigureaprecision4.Theclassificationof87%.hasanaccuracyof6

Discussion

FormuchtheANUcollection,theclassificationresultsbetter96%precisionforarethenearthantherandomWebcollection.successThewastrainingperfectandwithtestansetsaccuracyof98%andsetsis99%and95%fortherespectively.trainingandThetestablyrespectively.automaticallysimilar,therulesSincegeneratedallthesefiguresbyC4.5arereason-andtheabletosuccessfullygeneratedfeaturesappeartobefromrobustthecuracyTheANUrandomdomain.

classifynewexamplesfromWebclassificationobtainedanac-setsforcuracytherespectively.of92%andtrainingandThe85%forthetrainingandtesttestprecisionsetsrespectively.was96%andThe87%ac-slightlyandistolowerhigherprecisionthanthethanANUtheobtainedtestinthetrainingsetisdomainset.Overall,butthistheisbelievedsuccessfoundbedueofaretheinsearchthetomoreWebvariationintheexampleinterfacesinterfacescollection.foundAinsubstantialtheANUcollectionnumberterfaceactuallySothenaturally,ofPanopticthesametheANUwhichinterface,beingthesearchin-collectionistheANUisnotsearchasdiverseservice.boundrandomlection,onstillveryantheWebgood.accuracyclassificationcollection.Overall,takingthelowerasof85%forandtheprecisionrandomofWeb87%col-is7

Furtheranalysis

Wegeneralnowaddressthreefurtherquestions.testaretherulesinFigure3andFigureFirst4?howWeimentsthisfier?FigureThird,beenbycrossaffectedvalidation.4rulewhatonacanbylargebeourSecond,haveourexper-Weblearnedchoicecrawl?fromoftheapplyingC4.5classi-the7.1

Rulecrossvalidation

ThisgeneratedsectionWebfromcomparestheANUthecollectionresultsofontoapplyingtherulessultscollectionclusionsofthisandviceversa.Figure5showsrandomthere-ANUcancrossbedrawn.validationFirstlywhereitsomeappearsgeneralcon-Webwas76%collection.ruleshavelimitedontherandomThebestapplicationtothethatrandomtheWebaccuracytestcollection.fortheseThisrulesis

Predictedsearch

non-searchPredictedActualsearch

C4.5SVM145143C4.5Knn146SVM4Knn63Actualnon-search

C4.5SVM44C4.55

SVM66KnnKnn

6665

Table8:Otherclassifiers,ANU.

Predictedsearch

non-searchPredictedActualsearchC4.5SVM132C4.5Knn136138SVM18Knn1412Actualnon-search

C4.5SVM11Knn

23C4.59938

SVMKnn

8772

Table9:Otherclassifiers,Random.

significantlywithlowerthanantree.rulesgeneratedfromtheaccuracyrandomof85%WebachieveddecisionANUSosoarethisnotwouldrepresentativeindicatethatformsfoundonthethetherulesgeneratedfromofthisthelimitedWebatlargeandperformRulesWebconsequentlygeneratedwithhavelimitedsuccessexposureatlarge.toiswellontheANUthecollection.randomWebThecollectionaccuracymarginallyabout95%theworseandthantheusingprecision91%.ThisisonlygenerationWithANUtherulesgeneratedwiththesedecisioncrosstree.

validationexperiments,classificationstrategiesshouldresultsonemerge.aFirstly,forthetwobestrulegoodnobasedlocalperformancebebasedonlocalexamples,certaindomain,providingtrainingverytrainingdataoverthatisavailable,domain.trainingSecondly,canwheneralformancepurposeongeneralinmultiplerulesearch(suchinterfaces,providingagen-bedomains.asFigure4)withgoodper-7.2

Evaluatingotherclassifiers

WeentthatWekanowperformtionourresults(UW2001)thesameexperimentswithdiffer-werenotclassificationdependentschemes,onC4.5.toInensureandK-nearesttoC4.5weneighborusesupportvectormachines(SVM)addi-otherResultsareinTable(Knn).

8andTable9.NoneoftheC4.5.classifiersinhigherTableAlthoughprovides9forSVMtheandtrueaKnn,positivessignificantthisisareadvantagecounteractedslightlybetteroverbyaitivesleadtoarefalseaqueryparticularlypositiverate.beingundesirableInthisapplicationfalsepos-senttoanon-searchinthattheymightcouldPerhapsschemes.beachievedwithusingtuning,theseevenandbetterperformanceinterface.C4.5classifierHowever,hasperformedforthepurposesotheradmirably.ofthisclassificationstudy,the7.3

Real-worldtesting

Onetheoftheauthors,whohadnotinofPerloriginalandcode,reimplementedtheFigureworked4withtreepanopticsearch.com/Australianranresearchitoverinstitutionsa2.5million(seepagecrawlthenewscriptproduced).resultsAfterwhichminorhttp://rf.closelyadjustment,matched

SearchinterfacetargetsUniversity2+forms1+forms

anu.edu.auqut.edu.au

627

unimelb.edu.au5932cqu.edu.au1811653349latrobe.edu.au1705monash.edu.au874usyd.edu.au86rmit.edu.au6913610unsw.edu.au67515uq.edu.au

65287canberra.edu.au554327905uwa.edu.au

51624murdoch.edu.au48438uws.edu.au45curtin.edu.au38327gu.edu.au28deakin.edu.au26mq.edu.au2510775uow.edu.au22adelaide.edu.au1810082scu.edu.au1662csu.edu.au1423unisa.edu.au1363ntu.edu.au1340flinders.edu.au919uts.edu.au7swin.edu.au731927utas.edu.au613jcu.edu.au526acu.edu.au

479newcastle.edu.au4une.edu.au4209ballarat.edu.au312usq.edu.au3ecu.edu.au2167bond.edu.au1usc.edu.au1152vu.edu.au

10

24

Tableuniversity.10:TargetgetsForexample,URLsofofdetectedthe19337searchdetectedforms,bybytwo2653orweremoreatdetectedANUandsearch627offorms.

theseweretargetedtar-thoseingofTables6150testcodeforgenerationand7.ThefinalPerlscript,includ-212Oflines.

ofconfusionmatrices,was569theor40%530763wereformsflaggedinasResearchsearchFindercrawlForMorewhichexample,interesting2945searchthanformsformsareareforms.

detected,formtargets.allofanusearcher,(viatargettheirtheURLhttp://search.anu.edu.au/itthisisformaction).ViewedbyauserormetaNote,mightathisbealsousedoneallowsinsearchdifferentratherthan2945,althoughustogetwaysabydifferentforms.detectedURListargetedbybothdetectedmixedsearchview,formswhenFinderWefoundnon-search44744forms.

andformtargetsintheofcrawl,ofwhich19337or43%wereResearchtargetsbeingdetectedtectedtargetedsearchbybothforms.detectedOfthesesearch1563formswereandmixed,de-searchTablenon-searchsity.targets10isaforms.

appearedsummaryatofhowmanyofthe19337theItcountsthetargetseachofsearches,AustralianratherUniver-thansearch/index.aspsources.Forexample,itistheiftargethttp://www.uq.edu.au/URLofrateiscountcountedforonceformsinwhichTableare10.targetsWeinclude4681oftwooraforms,sepa-more

detectedAninterestingsearchforms.

caseisUniversityofQueensland(uq.edu.au),whichincluded55detectedsearchtar-getsonewithmultipleforms,butaare:

form.Thetoptenmostreferencedfurther4850searchwithtargetsonlyhttp://www.uq.edu.au/search/index.aspyes=4681http://www.uq.edu.au/search/index.asp?yes=1662

http://www.sph.uq.edu.au/search/sphsearch.idqyes=1180http://www.uq.edu.au/myadvisor/index.htmlyes=263

http://www.its.uq.edu.au/factsheets_search.htmlyes=234http://www.commerce.uq.edu.au/cgi-bin/htsearchyes=104http://www.its.uq.edu.au/faq_search.htmlyes=77http://www.uq.edu.au/myadviser/index.htmlyes=57http://asc.uq.edu.au/gradnet/main.phpyes=56no=1

http://www.library.uq.edu.au/uql/cgi-bin/subjectsearch.plyes=27

allofwhichtheThewhereformleastarereferencedsearchinterfaces.

UQtargetsincluded4065ofappearNNNNNNNhttp://student.uq.edu.au/~sNNNNNNN/afewformonbuttondisclaimerisastudentnumber.Theseforms,toproceed.pages,askingBecausepeoplethetoclickonnodeotherinFigurefeatures,4.

theyreachtheleftmostformsSEARCHhaveTheworstcaseweexaminedwasANU(anu.edu.au),withthetoptentargets:

http://search.anu.edu.au/anuyes=2600no=345http://search.anu.edu.au/externalyes=660no=65

http://tux.anu.edu.au/twiki/bin/search/know/yes=65http://netserve.anu.edu.au/commpro.htmlyes=36

http://msowww.anu.edu.au/cgi-bin/htsearch.wrapyes=35http://arp.anu.edu.au/arp-cgi-bin/escyes=31

http://law.anu.edu.au/legalworkshop/lwscripts/(none)yes=23http://tux.anu.edu.au/twiki/bin/search/twiki/yes=22http://tux.anu.edu.au/twiki/bin/rdiff/documentation/

webstatisticsyes=20

http://tux.anu.edu.au/twiki/bin/view/documentation/

webstatisticsyes=20

Theyinterfaceincludeberincludedofinterfaces(http://search.anu.edu.aumixedjudgmentsonadefinitesearchwhichfallfoulofthe)andanum-inFigureUQ4).

studentpages(leftmostsameSEARCHissuenodethateasilyWematedimplementedconcludethat,andalthoughallowedtheFigure4rulewasrequiredoverviewofitstype,furtherthefirstworkeverwouldauto-searchspiteinterfaces”todiscoverarewhichofthe19337“detectedbesuggeststhis,thelargenumberactuallyofdetectedsearchinterfaces.searchtargetsDe-ResearchthataccessFinderametasearchsearchengineproject,project,parallelingthewouldtobeplentyausefulofstartingsearchinterfaces.pointinsuchTheanlistwouldendeavor.of19have3378

Conclusion

ThissearchpapercisiongorithmtreeinterfaceshasshownwasdevelopedfromhowasettoautomaticallydiscoverwithofHTMLtheC4.5forms.learningAde-thecuracyHTMLusingmarkupautomaticallythatcangeneratedfeaturesfromal-asAnofabout85%forgeneralgiveWebaclassificationinterfaces.

ac-toCallanobvioussearchinventthe&Connellnextstepfirstfully(2001)istoapplymethodssuchautomatedorIpeirotissearchetengineal.(2001)stitutionsengines.engines,inatefalsealthoughshowthatOurpositives.furthertheretestsinAustralianresearchforin-workaremanywillbecandidateneededtosearchelim-References

Callan,plingJ.&Connell,M.(2001),Query-basedsam-onInformationoftextdatabases,Systems’,inVol.‘ACM19,pp.Transactions97–130.

Callan,J.P.,Lu,Z.&Croft,W.B.(1995),Searching

distributedcollectionswithinferencenetworks,in‘SIGIR’95’,ACMPress,pp.21–28.Chang,C.-C.K.,Garc´ıa-Molina,H.&Paepcke,A.

(1996),‘Booleanquerymappingacrossheteroge-neousinformationsources’,IEEETransactionsonKnowledgeandDataEngineering8(4),515–521.Chidlovskii,B.&Borghoff,U.M.(1998),Query

translationfordistributedinformationgatheringontheweb,in‘InternationalDatabaseEngineer-ingandApplicationSymposium’,pp.214–223.Craswell,N.,Bailey,P.&Hawking,D.(2000),Server

selectionontheWorldWideWeb,in‘Proceed-ingsoftheFifthACMConferenceonDigitalLi-braries’,pp.37–46.Craswell,N.,Hawking,D.&Thistlewaite,P.B.

(1999),Mergingresultsfromisolatedsearchen-gines,in‘AustralasianDatabaseConference’,pp.1–200.Dreilinger,D.&Howe,A.E.(1997),‘Experiences

withselectingsearchenginesusingmetasearch’,ACMTransactionsonInformationSystems15(3),195–222.Fuhr,N.(1999),‘Adecision-theoreticapproach

todatabaseselectioninnetworkedIR’,ACMTransactionsonInformationSystems17(3),229–229.Gravano,L.,Chang,C.-C.K.,Garc´ıa-Molina,H.&

Paepcke,A.(1997),STARTS:StanfordproposalforInternetmeta-searching,pp.207–218.Gravano,L.,Garcia-Molina,H.&Tomasic,A.(1999),

‘Gloss:text-sourcediscoveryovertheinter-net’,ACMTransactionsonDatabaseSystems(TODS)24(2),229–2.Hawking,D.&Thistlewaite,P.(1999),‘Methodsfor

informationserverselection’,ACMTransactionsonInformationSystems.17(1),40–76.Ipeirotis,P.,Gravano,L.&Mehran,S.(2001),

‘Probe,count,andclassify:categorizinghiddenwebdatabases’,ACMSIGMOD30(2),67–78.Lawrence,S.&Giles,C.L.(1998),‘Contextandpage

analysisforimprovedwebsearch’,IEEEInternetComputing2(4),38–46.Perkowitz,M.,Doorenbos,R.,Etzioni,O.&Weld,

D.(1997),‘Learningtounderstandinformationontheinternet:Anexample-basedapproach’,MachineLearning(toappear).Rasolofo,Y.,Abbaci,F.&Savoy,J.(2001),Ap-proachestocollectionselectionandresultsmerg-ingfordistributedinformationretrieval,in‘CIKM’01’,ACMPress,pp.191–198.UW(2001),‘Wekamachinelearningproject’.

AfreesuiteofmachinelearningtoolsavailablefromtheUniversityofWaikata.http://www.cs.waikato.ac.nz/ml/.Yuwono,B.&Lee,D.L.(1997),Serverrankingfor

distributedtextretrievalsystemsontheinternet,inR.Topor&K.Tanaka,eds,‘DASFAA’97’,WorldScientific,Singapore,Melbourne,pp.41–49.