Advanced HTTP Activity Analysis 2009 Goal The goal of this training is to get you familiar with basic HTTP traffic and understand how to target and expliot it using X-KEYSCORE g Agenda What is HTTP stands for Hypertext Transfer Protocol and it s the primary protocol for transferring data on the World Wide Web Why are we interested in facehonk myepaee nm i place- Iu-r friend Because nearly everything a typical user does on the Internet uses HTTP @meitru Why are we interested in i Almost all web-browsing uses HTTP Internet surfing Webmail YahoefHatmailiGmail etc OSN FacebeoklM yS pace etc Internet Searching GoegleiBing etc Online Mapping Geogle Maps Mapquestietc How does HTTP work HTTP is comprised of requests from clients to servers and their corresponding responses Many are already familiar with the terms client-to-server or server-to-client collection also referred to as client side or server side collection How does HTTP work A Client is usually referring to a Browser like Firefcx or IE which is also referred to as the User Agent The Server can also be referred to as the web-server or origin-server which is the machine that is storing the data that is being accessed like a web-page a map an inbox etc HTTP Activity 2- HTTP activity comes in two types Website cum requests Samar Server-tn-Client i I responses Client an User HTTP Activity is HTTP activity comes in Me types Website cem - Sewer Irttar'riaf Hit d 2 I While there may be a variety at Praxies #7 Gateways er Tunnels in between the client and n the server traf c is always geing in one directien Ghent er the ether User Client vs Server Side Traffic in How do you know which side you re looking at Client-to-Seryer requests are generally small in size and are computers talking to other computers They contain standard HTTP header fields like Host Accept Connection etc HTTP Activity Examples CIient to-Server request 5 HtetLLIrgJer HUD-GET 1 4 PrmtE Flewdl' nareinn HEWLEH ENIFurmat i Service vi HI 1 11- LJSJ pplu lLi' hlL 111 1 14 5H Ea am l gna n Tm - r-n I- Startli tm F flint-apt 3cm '9 '5 zcupt ne-3 121 g p c nteb plsich 1p use i n pt-Tanglag ' zcupt Chant ISO E3551 Lit E 3 Heel amaze-1' Etc-m Czarzustli'rn Resp-EJqu Client vs Server Side Traffic Server-to-Client responses are generally larger in size and are what web-pages look like at the internet i When you re at a computer accessing the Internet you re only seeing Server-to-Client traffic HTTP Activity Examples Server-to-Client Response TI Document Information r- - _nrig_pl Flaw Fr ms Bonus question are the images in this web-page missing 3 llT'l'F' Ila ads-I Il1l fl1l ti -l1 I SH ruluub r5 1 le'ui'ill 5' it-LI- Id twur 11 41 PEEK ir i Kuwait gownimrnt resigjls imam 3'33 Mn Mala-11F i r Hllwaltl qnu has or In i1 LuL-sz-n rssiyndliun to ho urnir dmitl run-1' Tux- over the premier'i hon ofthE economic I'tl i' is E'Irrsm' _1t1lF 3 i met gnaw SE rue-Slc'iazlol' has SUJInlztE-cl Fc'ma and 31 I - I Litrue smll' rule-r to F's-Hrs - I Iu'l I I _ iim if nil-nor Homan on nmr-nr 1mm Lust pics-3L1 l 1 Irl 1Jr'l Ii 1r r_ r l h rltu' 1r-nrr ml 51 no jll lul E i1 _tg Ann- 1 took - Emma billion rusooa wl' sh 5 to the F r3 an Gull 'IstILr H fi Il cling to ease the nft'iE- global coils d T r gr mt l' 1- no 1m rtr'rrl 1 l1 1 I Suzi-Tn sh 2 31 11 HTTP Activity i XKS HTTP Activity Meta-data differs greatly depending on which side of traffic we re collecting i In nearly all cases it s better to have client-to-server traf c TT I a I we a Accept Retarer 1111 1313 f aearch hhc an q musha atastatt 3ascnye urdu Accept meage Accept Em - - User-Agea Maxillaf4 D compat h e HSIE 6 0 Hindaws NT 5 1 Svl mmnma Ennkir Willi-3 El KEE K-Eluetuat Via Hnat UHL Path URL my Eirawaar campa bla MSIE anwa NT 5 1 Sim Search Terms Language 'au ia muaharraf an Hafarar i111 ht C uae 3530 2133 acb Elia-45196 afaEIS-ll a 9 20932 HTTP Activity Server-to-Client 3 r Iflrl Fir-155 - Huwal gumz-I'nnh'mt 'I'uzealgns - 'lm- hem - I'Irl IIE-IIJEI 'IT'r'n- Tat - Lia-1 1 1- anl' ersL-u' lif liliwuil M'nrl' - lune tau-Ha tI has suhn'u'ttn- rt in I'll- I n'nill 1 'Ill'J pr a tl'l EE -l an li-Z I'n'l 1 en - - 55111 Ti' Hut-r r u I- l HT Elu 1 Hr - run at l l' HTTP Activity HTTP Types i Meta-data will also tell you which side of traffic you re looking at i Client-to-server has two main types Server-to-ol' I Type only one HTTP Activity Get vs Post i A is you requesting data from the server most web surfing EA is you sending data to the server signing in lling out a form composing an E mail uploading a le etc Let s break down the important parts of a client-to-server request HTTP Client-to-Server GET i'hnn ie ht Heat User Agent Windnwa Ll Windews NT 5 1 en US ruz i 3 9 10 25 Firafnx tifi El Accept Accept-Language Aacept-Eneading gaindaflate Accept Charset Heep Alive BUD Connection Heep-alive First thing to note is the Host line which tells you the name of the server that the client is requesting data from Host Field It s important to note that in many oases users think they re at websites like but behind the scenes data is coming from a number of different servers without the user knowing it 1 si'i z a till1 11 3 mail I nit-xi - 27- Iii-aims - sEu - -i E r5 Ezjlter-Ir a t En gag - Itk at -rI -t-lET some - 5 Tl 32-1-5 Bonus question What would the impact of this be in how you formulate your queries using the Host field GET theme- Heat samplewebsiteceh i User Agent Mezillai t'ti'u'inclews U Wir1 dews NT 5 1 en US 25 Accept Accept Language Accept-Enseding gzipldetlate Aceept-Charset Heep Alive EDD Cenneptieri keep-alive Second the GET line tells you which files the user is requesting from the server If you simply take that line and append it to the Host line you have the live public URL that the user is reques ng GET Host samplewebsiteccn r User Agent Mozillai t'ti'ii'inclows Windows NT 5 1 en US 25 Accept Accept Language Accept-Encoding gzipldetlate Accept-Charset Heep Alive EDD Connection keep-alive When the GET line has a mark in it then the GET request is also passing information to the server So in this case the client is requesting the file examplephp but it s also passing along a value that could have been entered by the user URL Lines When there is a mark in the URL line then KEYSCORE is breaking it up into two parts The first part is called the URL Path and the second part is called the URL Ar ument Path Large f5 El 3 tE-l r' r th IZIJE E ha rail-5 at 31 rt El ELSE IZI Ll Elli Notice all of the arguments each separated by 863 in this RL GET Ref-am 11 try ear-2h tube can eat h 1 31 if aata ccEpt-Enc uzliztrg 2 in defl ate BDHUS Any idea What the ur- infermatic-n that is being passed in the RL Argument In this example are far an Keep it ice Ell E- 9 1153 5 46 HTTP Client-to-Server Hoet earnlewebeiteoom User-Agent i'u'lozillai' ii j ii'ii'indowe Windows NT 5 1 en-US noi gi i GeoirtoiE 9042315 25' Accept Encoding gzipoeflate Aooept Chareet Keep-Alive EDD Connection Keep alive The User-Agent line gives you information on what type of client is requesting the data In this case we can see that it was a Firefox 3 0 browser from a Windows NT 5 1 XP machine User Agents User Agents The UserAgent also known as the browser can be very valuable While it can not be trusted to be absolutely unique in many cases you can use it to unwind a proxy or multi-user environment It can also help provide hints if the origins of the request came from a mobile device TLII Cl Fig-lequ-t CID 1 11E at 111w titdiu irio t5 E 1' 51 - 11- - 1 1 Us or- Agent Elton 1 HTTP Client-to-Server GET themehtml Host samplewebsiteccm User-Agent Mczillsf Windows NT 5 1 en-US 25' Firefcsr'l ft Accept Accept-Language Accept Enccdirlg g p eflste Accept Charset Keep-Alive EDD Ccnnecticn Keep struts The various Accept lines instruct the server on the types of responses the client can accept back Let's look at a simplified version of a HTTP request and response What is Web HTTP Activity This shows hew a person legs en to webpege Fi m 3434 Click on TD 3 3 client GET Request SEWEF The elient e pert een be an high-numbered pen 3434 is just an example What is Web HTTP Activity This shows hew a person legs en to webpege PW 3434 Click en http whetmeileem 80 client GET Request sewer 4 it Frem Part 813 Ft 3 34 Weleemete Hetrneil Sewer 3 Reepenee The elientl e pert can be any high-numbered pert 3434 ie an example What is Web HTTP Activity This shews hew person legs en to webpege Fr m PW 3434 Click en 3 3 Ghent GET Request Sewer Tm lam-t 3434 Frem Pert 8D client Weleeme te Hetrnell sewer Respense client EmeilAddress me@hetmeil eem Sewer Password Ad min123 POST tn the Web server The client s pert can be Eil'i y' high-numbered pen 3434 is just an example What is Web HTTP Activity This shows hew 3 person legs en to webpege Fi m 3434 Click eh 3 3 client GET Reque t SEWEF P rt client Weleeme te Hetrneil sewer Respense client EmeilAddress me@hetmeil eem Sewer Password Ad min123 POST tn the Web server Te F ert 3434 Frem Pert ED CHEW Welcome te yeur Inbexihemepege it HTTP Respense The client's pert can he anyF high-numbered pert 3434 is just an example HTTP Activity i Real traffic however can be a little more complicated 1 Almost all web pages are built from mumme bs For example every single image or banner ad on a web page is a separate file that needs to be individually requested before the server that has the file can respond HTTP Activity Real World - Let s look at the Today home page ' lfrA-rtilflT 1- E El I - il'i - rm rrent Gandhi-ans fl ITTI 110 HE- uwu il spend Indian j Day-5 m Ln HHarr ' l-rl lune I Hi In THE - HERE- Hc-Et the EENTE Ell-J1 Eenler Enlleted Leader LI I F'rpm If E- 5 Eti'llj-Z'E'l'i Lrl'r'ly I I - I 1 Ir' I nr I12 II II 'rf II I I Il'ns- fir-5t l' FIT f lr 1 H- ll'i'E'r'u'iE'Tl' tE 3le lL-Itl - 1- Id LI eel HTTP Activity Real World - It looks like one page but each of the images and banners are separate data files that your browser pieces back together - -- I LJIF fit-LID I asrs 111a GEE-TECH ram-1r Enlisted Lea-jar - -J - The ND 5 i'E 0' ND Ed the en Iriulill lrl r r L LI iI_ I I 'l I hri I I HTTP Activity Real World In fact to build the NSA Today home page it takes 34 separate les from 4 different servers However most people probany don t notice because the entire page loads in 300 milliseconds i If we had a slow internet connection we'd notice the images would initially be missing TI Datum-ant Flaw at-1a 1'1 HTTP Ila ails-I Il1l fltl ti -l1 HTTP Activity Real-Word Netiee that all at the images are missing They are all separate sewer-te-elient responses and therefare sempletely separate sessions in X-KEYSCORE er PINWALE Fr mal SH raises l-jsmr rs i -fltiia I-Iaatt Palestine Let-Luau Turks E'ersm' 3an Others afhra Litu'ap as Sci-Tech saltll lild ever 11 31 Latest News Kuwait I'esigns nt rl' tummy l nl gr Eu Thine-cut 7 air 1'3 The Hllwaltl has suhml'rterl Its resign-alien the LIIJL-ll'll'f'b ernir dlrlitl ever the premier's han efthe economic nrisls I i duel-t3 ms I 3 tits-int E1 Il's- resign-32hr has teen susmittesl Firms at it Lp the emii' ruler t nil-itnr mnn' 1 sir-r imr-nr tr-rm i-fLIrI-Jsgr If Leas 1 part-lent r'I't TJr'Ilt'n Zitlt'f I'Ir'l paekage wt' eh 's la the Far - Gull 'IstiL 5 en ul'am't- eass the impact nft'ie gletal 'Is'isncis' Tar rat 'in' Int m tr rrl in tin -r 3 11 et HTTP Activity Real World i It s important to note that not all of the data on one web-page came from the same server i For example most of the NSA Today home page come from but the image of the current weather conditions came from wk- admiral208 corp nsa ic gov HTTP Activity Real World i This happens all the time on the Internet i The cnneom home page may have an ad on it that was from the Google ad server and etc i And this does have an impact on our collection i This is the traffic path for building the NSA today home page I I 1 - heme warm nsa 53 aitewerkanea we admiralE B eerp neaiegm i What happens if we only have collection on one of the paths 2 I thE uWM nsa erwem 53 altewerkansa wk admiralE B cerp neajcgm Emma 0 What would that traffic look like GET Heat wh edmirelE E eerpneeiegev User Agent Mezillef Windewe Windewe NT 5 1 en US W11 3-6 10 25 Firefem B fl Accept Accept-Language Accept-Encoding gzip deflete Accept Chareet Heep Alive BUD Cenneetien Heep-elite lf Medified Sinee Thur US Get 2009 19 31 56 GMT lf-Ne rte-Match 19454 Ee i 42dh643 Bethe-Central max-agem If we enly saw this ene GET request and net the ether 33 required te build the NSA Tedey heme page weuld we be able te determine what the user was actually doing What exactly is that telling us iv First off we know what file they are requesting aThey want current ij from the wk- admiral208 ccrp nsa ic gcv server is That s actually a live public URL in Do we have any indication why they wanted that image Answer is yes Look at the referer field What exactly is that telling us in They were referred from Iii-The referer is in essence telling you what site was linking to the new site in Warning The referer can act in misleading ways Referer Field The referer field is the address of the page that links to new GET request However this link could have been automatic to the user i Le in the case of the current weather image the link was automatic and the user wasn t even aware of the action Referer Field in The referer field could also indicate a user ac on For example imagine we were on the NSA Today webpage and clicked the link to the SID Today page What would that traffic look like Referer Field I Host sidtndaynsa Use r ige n t Ha zlf'lafB Windows NT 5 1 en-US Gad- 02009042316 SIS-25 Firef x U 1 0 Accept Accept-Language Accept-Encoding gzip eflate Accept Charset ISO-38594 Keep Alive 300 Referer t3 EFT GEE 66534 96 eated%3D x 7Bt5%20%2T2009%2 itcnu nt%3 Referer Field it Now we re seeing a request go to host sidtodaynsa with the referer from Ir How can we tell from the traffic that the first automatic referer we saw for the current weather was any different from the user- generated referer we saw for the SID Today article Cookies Cookies '1 Cookies are small pieces of text-based data stored on your machine by your web browser I Almost all websites have cookies enabled and they have a variety of uses including to help the web-site track the activities of their users i Most are probably familiar with machine specific cookies like the Yahoo cookie I However cookies are used for a variety of reasons What can cookies be used for Cookies can be used to authenticate a user For example in many cases the active user for Yahoo web-mail traffic is seen encoded in the I part of the cookie string I ugliele i til-111111131 Unite- 1 Eta What can cookies be used for Cookies can be used to store information about the user that the website is interseted in Look at how the p value below tells the website information about the user of this account I ugliele ji l' ill-111111131 Unite- 1 Eta 'l What can cookies be used for i Cookies can be used to identify a single machine from hundreds of other users on the same proxy IP address It The Yahoo cookie is a machine specific cookie What can cookies be used for Important note All three of those examples are just subsets of the full Yahoo cookie string HOW do we EHOW wlla 680 COOEIG value is used for i Nearly every web-site uses cookies that in most cases they designed for their own uses so how do we know what they all mean 1 Protocol Exploitation can examine the traffic to try to determine if there is any information contained in cookie strings that we might be interested for example we d like to know if any part of the cookie acts like a machine specific cookie HOW go we EHOW wlla 980 COOEIG value is used for it However there are far more cookie options out in the wild than PE can possible examine 80 even if they aren t aware of a machine specific cookie it doesn t mean that it doesn t exist X-KEYSCORE gives you access to the full cookie string so if you re adventurous enough you can do your own protocol exploitation Remember Cookies are there for a reason in Websites put cookies on people s computers for a reason If the data is valuable for a website it may be valuable to us as well How long do cookies live for Cookies like any other file on a computer can be deleted by the user I Almost all browsers give you the option to View manage and delete your cookies awn-a Cookies You can see whet have been stered en your machine by geing inte the eptiene windew ef yeur breweer and selecting shew Dptinrts Meir etI-s crtent ice ien-s lu'l I_l JL II i F_Ener uer I in ern'E anzl the bar Frl r 'Er' IEI Jag- 2 i fr- site rte esp-std Iths Emir-e r I I er Lia-cad 54 I Searches Searching the Internet When a user searches the Internet from one of the many web-based search engines Google Bing etc what does the traffic look like Searching the Internet CIient-te Server In most cases the client-te-server traffic is a GET request where the search term is passed in the URL Arguments GET HTTPI1 1 Heat wgeegtesem Assent imagei gif imageht-xbitmap imagefjpeg imagei'pjpeg applieatienfmewerti Beckie 4 3 K5hp45ri h Accept-Encoding gzip deflate User Agent Mezillafdl issmpatihle MSIE 5 0 Windows NT 5 1 Cennestien Keep-Alive Cache Central ne eashe Searching the Internet CIient to Server Notice how the URL Path is lsearch and one part of the URL argument is q iran 1 Each website can configure their differently so while with Google the search term is contained in the q part of the URL a different search form might have it as query or search_term etc Searching the Internet Client-to-Server X-KEYSCORE tries to account for all the variations of search terms contained in the URL Argument for what it extracts for the Search Term column it However there are always other varieties out there that we haven t built it hooks for yet so anytime you see something that you think should be extracted please contact the team Referer Searches in What happens when a user on a search result Let's start by showing the query itself in this example we're going to query the Goegle for Referer Searches What does that GET request look like GET Heet geegle4 q nea User-Agent Mezillat Windeweg Windewe NT 5 1 en-US GeeketE Q - i i Firefexi ji Accept Accept Language Accept-Eneeding gzipee ete Accept Ghareet Heep-Alive EDD Cenneetien Heep attire We knew frern this seseien that the client is requesting the data fren'l the heat geegle4 q nea and we see the search term in the URL Argument Referer Searches What happens when a user clicks on a search result GET I'recln'line Heet r1 nee User-Agent Mezillef Winnie-we Windewe NT 5 1 en-US GeekefE DQ - t t Firefe S JE Accept Accept Language Accept-Encoding gzipeeflete Accept Ghereet Heep-Alive 3G0 Cenneetien Keep alive Ceekie Referer First we can determine the full URL I by adding the GET line t0 the heat r1 neefred mine 3 5 Referer Searches i Secondly we get some hints as to why the user was requesting that page from the Referer line Ftete re r g nsaisea rch d xkeys ccre htnG Gccg le 3ea Note that it was the same URL that we were at immediately before we clicked the result link Refersr Searches Let s look at that process again r nss First a slisnt to- rsqusst ls sent that contains the s usr1 r on Referer Searches Let s look at that process again 31k ey oorers r n a googl q n33 Second the server responds hack w h the search results Referer Searches Let s look at that process again MLIU oocgleeq nse I Ikeyacore r'i heal Third by clicking on one of the results a newr GET request is issued to retrieve the home page In this request the location of the original search is listed as the referer Referer Searches it Let s look at that process again googleeq nea Flililil Ikeyeccre r'l nee What will happen if we only have collection on this link Referer Searches When XKEYSCORE sees a search contained in the referer field we still extract it out as meta-data into the search terms but we append it with referer to denote where it was originally found E'earor' lreferer the legal status oftlle caspi n EEJII F'Iritl'l F-ief er r' -r ll E'hl f Ea-Lilli ue 1-3 IE-F-Fil lliillHE- al ll Referer Searches GET Amer Has Rafe- er Accept-La nguage fa Accept Enc di ng gzipb deflate Llsar-ngnt cmnpat'tble MEIE Fit INin-d wa NT 5 SW HET ELF EELEDTET Cache-EC ntml Cannecticun chase I-E- ilua mat-W51 Can we guess what happened here Referer searches Another example 5 HtetLLIrgJer HUD-GET 1 4 PmtE- Flewdl' narainn HEWLEH ENIFurmat i Service vi HI 1 11- LJSJ pplu lLi' hlL 111 1 14 5H Ea am l gna n Tm - r-n I- Startli tm F Elmer-t 3cm '9 '5 zcupt ne-3 121 g p c nteb plsich 1p use i n pt-Tanglag ' zcupt Chant ISO E3551 Lit E 3 Heel amaze-1' Etc-m Czarzustli'rn Keep-EJqu Proxy Information Proxy Information In a lot of cases we re going to see HTTP Activity from behind a proxy or proxies What is a proxy A proxy is a server that is acting as an intermediary for HTTP requests from clients Why do proxies exists - Performance Proxy cen csche responses for static pages - Censorship Proxy sen - Security Proxy can look for msiwsre - Access-Control Proxy can control access to restricted content Proxy Information in Routinely we re going to see ISP level proxies That is instead of having each individual user request web pages directly from the web servers the ISP is going to collect all of these requests first and then proxy them out through a handful of proxy IP addresses it When the response is returned the proxy passes it on to the appriopriate user Proxy Information in Why would the ISP want to proxy traffic In many cases the ISP won t have to supply public IP addresses to all its users it It can simply give them a private IP address and then use a handful of public IP addresses for its proxies which are the machines actually requesting the traffic from the web-servers Proxies on the Internet H Web Servers Web Servers bl l E Ii- l l Multiple-ueere multiplexed Web-Sewer Shelf-lived cennectie I15 Single-neer LID ng-lived l ll l-Etti t l Multiple-ueere multiplexed l Identifying a Proxy It'- How do you know that the IP address that you think is your target is really a proxy First step check NKB They have services that attempt to automatically detect proxies These services are in no way 100% so this is only the first step in oheoking to see if the IP Address is a proxy Identifying a Proxy NKB Query am ute EDDQ- Cnntidcnm 'n'alu I I I II A11I I P 1Ii1ll rr 1'l A F I pr'E I - - - EZIZZEZEE - - Elli-E JT an I ma '3 T four- 1 - I I Identifying a Proxy in Other things to be on the look out for i X-Forwarded-For IP Address What is it An X Forwarded-For IP address the proxy passing on to the server what it thinks is the IP address of the user Think of it as the proxy telling the server this is who I think this request came from It s important to note that multiple proxies can and often are present so one proxy mightjust be reporting the IP address of another proxy Identifying a Proxy -1 X-Forwarded-For IP Address as seen in traffic GET 1 User fag-ant atibls r133 Ell aw-3 FIT fl 1 5 171 3 st imam-v Iii - 513 - I 3 3 I I e1 - L 1-1 1 1 2111 I LIL l - 11 1 au- c1 17 '13 a the '13 rim 5th 5 ill Ii - r1111 ii-1 311 L a slurs Some Examples of X-Forwarded-For headers lit-Ferwartt Ed- Fer K-Femrarded-Fer H-Femrarded-Fer K Ferwarded-Fer X Fen rardedd Fer K-Fen rarded-Far 12100 1 - K-Fewrarded-Fer Multiple-Layers ef Pruxieel ln-general the first IP is the one to the original requester Keep in mind these can be tetally fake Identifying a Proxy Similar to the X-Forwarded-For Tag is the tag The VIA tag is the proxy identify itself GET II H51 Um ht Ht 3t E tit le I-JEJIE- rt - 3 231 71 II 1 EIF 'Cl' l 3 Elm til I I I Iii F i - 1 ESE 35 1'35 E1 the - 111- 51 E in I I 21111 1-3 EFI -Elhn Identifying a Proxy The Via tag may even contain some good information about the proxy It Be careful though because this information could be falsified CI to hr all-1 1' '32 1 El Iii-3 5 Identifying a Proxy - Remember though that the X- Forwarded-For and VIA lines can be falsified and don t have to be present Ifthey re not present how can you tell the IP address is a proxy Test it in Testing IP Addresses in MARINA r- The primary side effect of a proxy is too many users online at the same time So if all else fails try querying on the IP address assuming its USSID18 compliant of course in MARINA to see how many users were active within an hour time frame it It s not scientific but generally it will help Testing IP Addresses in MARINA For example look at these results 1-3 311- liming in 1 DEE In Ile Eire E121 1- I Hate T 3qu 411 Hat by El that I rT-Eltct'l the 1 7 111it There were 274 unique Active Users in that hour think it s a proxy HTTP Header Fingerprint HHFP What is the GCHQ created the HHFP to help identify individual users behind a single proxy IP address i The HHFP is a hash of multiple header elds that can be used to identify a single user behind a proxy What is the in At least one of these values must be present X-Fomarded-For IP Address Via Client IP address Ifso the HHFP is a hash of those values combined with the User Agent string I ll-l El r31 Erit late a '1 5 Era- 5 EX Here s 8 'rania WOW Address that has multiple underneath it '1 i1 i 3 l a 5 f NOTE There s no guarantee that an HHFP is identifying a single 33 2315 3323 53513-15333Ijj ljiEri-E El austere 13 that more than one user will have '3'1 1 E- 4 5 321 th a 1 45 3 4 it t Ei'i-E at 0 21 12 aux- 1 _i 51 34 11 911al'l'i 4 Pros and Cons of HHFP I On the positive side the HHFP is a single 8 digit value which can help identify a single user behind a proxy On the negative side it requires an XFF IP address Via string or Client IP Address and since many sessions do not contain all three they ll have no HHFP string Also even with the HHFP all of the fields that are used to build it are available in the XKS HTTP Activity query so it s not providing you with any data you don t already have access to HTTP Activity Search XKS HTTP Activity Search After that overview of how HTTP Activity works let s look into how to effectively target it through XKS queries XKS HTTP Activity Search HTTP Activity indexes every HTTP session i CIient to-server and server-to-client i Can be queried on any of the unique HTTP meta data fields or any of the standard DNI fields IP Address SIGAD CASENOTATION ete XKS HTTP Activity Search Unique Meta-data fields of this search Include already cavered In training HIZISE 1-1 Fn Fail-I - Draw HaSh Tar-Fa Lari-Juana 'Dl rm ttauzl'rn-al'lt Tm 91 er Twin LhararerEnIdnq FEM Fil MEI l- j Ctr tart Etzup 3 NEWS Car ant T333 I XKS HTTP Activity Search - In addition to all of the common fields like MINES Fm Appleattoe Into IF'etljl'e55 To FEIT Flor-u rn ul'l l r'r'l' Counth i -rofr Tr I mite l Segeior lLEHUth men BMW Ramiro ll'l XKS HTTP Activity Search it Most commonly HTTP Activity query searches in XKS will be to enable persona analysis i Based on TRAFFICTHIEF or PINWALE we ll want to query XKS to discover all of the HTTP Activity that occurred around the targets session of interest Simple HTTP Searches In order to do a persona analysis type search all we ll need to fill in is the IP of the target assuming it s USSID18 compliant and a short time range around the time of the activity Datetirng I Il XKS HTTP Activity Search Another common query is who want to see all traffic from a given IP address or IP addresses to a specific website XKS HTTP Activity Search i For example let's say we want to see all traffic from IP Address 1 2 3 4 to the website i While we can just put the IP address and the host into the search form remember what we saw before about the various host names for a given website Host Field It s important to note that in many oases users think they re at websites like but behind the scenes data is coming from a number of different servers without the user knowing it E J's-1' El 3 1- II emit image 1 11 5 mail I nit-3t - 27- Ttat - -i E r5 a with 13L pz- En Lining gag - I'Jt l one an 217 I '4 fit11 1 5 i_ 1311 gill Gentle-E malla-i Eirtlt year Peatal XKS HTTP Activity Search i In order to account for all of the possible host names we must front-wildcard the host name i Be careful when front-wildcarding because beyond being resource intensive for XKS it can be dangerous from a perspective Hints for wildcarding a host name i If you re trying to query for traffic to the website the best way to wildcard it is website oom Notice that the before the hostname website is still there that way we will properly hit on ads website com images website oom but avoid the false hits on Hints for wildcarding a host name Why are we only interested in traffic coming from our IP of interest going to our website of interest Helpful GUI Shortcuts Earlier we talked about how XKS broke 3 GET request into the URL Path and URL Argument separated by a EX http forum Get s broken out to HIZI Eft LI FCL Ell l'l 15 f-crrum IslIcr'n'lhreathillp #131435 Helpful GUI Shortcuts 80 if we were to query for this URL we would need to enter these fields in separately He at URL Path rge #131435 Ho's-t tun-I r11 E-rgs # 1314135 Helpful GUI Shortcuts - Or we could use the Field Builder to simply copy and paste the full URL and let XKS break it into its appropriate parts HIZIEIZ 3 I l l liar-gs Field Builder that m It lurk-ELI I IJ mpululu Inn-L path and argument fields ritar Helpful GUI Shortcuts URL Field Eluitler Enter a URL that Imull he paried tu populate the best path and argument fields Ell 1 314 35 El 5t ll rum LII-7 1- atl' rt-1 1 re id F3 hp F-LL