A note on JSONP & misconceptions of Cross Origin AJAX

Web developers who have worked on accessing APIs using JavaScript would be very much familiar with the term “JSONP”. Many web devs whom I have met offline or in online discussion forums seem to have some misconceptions about JSONP. Below are some of the basic & common definitions which I have come across:

  • JSONP is a technique to work with remote APIs
  • It is nothing but Cross Origin AJAX
  • If we add a query string like “?callback=someCallback” and fire jQuery’s $.ajax or $.getJSON, what we are doing is nothing but a JSONP call.
  • May be a slightly complicated definition: Cross origin AJAX is possible only when the response thrown is JavaScript
  • and many more..

The truth in the above statements is very little and such definitions add more confusion, bringing in misconceptions. In my recent presentation “Content Isolation with Same Origin Policy”, I put up the below slides (check slides 4 & 5 in the ppt)

image

image

 

 

 

 

 

 

 

 

 

 

For all practical purposes, the first one is possible and second one is not. Apart from the tweaked definitions of JSONP as stated above, the below reasoning complicates the topic:

  • In the first case, the content requested is of the type “text/javascript” while in the second case it is HTML. So browsers look at content type of the response header and decide whether they should block the content or not (actually, a very good observation).
  • There is a “?callback=?” parameter in the first case enables jQuery to make the cross origin call in the first case
  • Server side framework should have special capabilities (Iike inbuilt serialization/deserialization) for the first case to work

I thought it would be nice to summarize few facts and hence this post. Read on.

What's an Origin?

The combination of scheme://host:port is what browsers treat as an Origin. e.g., http://abc.com, https://abc.com, http://abc.com:81 belong to different origins as they differ in one of scheme, host or port. Remember that http://abc.om/user1 and http://abc.om/user2 are different URLs but not different origins. Also, a domain (http://abc.com) and its subdomain (http://sub.abc.com) belong to different origins (this particular restriction can be relaxed using a technique called domain relaxation, which is out of scope of this topic).

Can my client script read your emails?

Browsers restrict JavaScript calls to server (read as AJAX) based on Origin. This is governed by a policy called Same Origin Policy. In other words, client script in your page can make calls only to your server (strictly speaking, origin). If this rule wasn’t there, it would have been possible to write a script in some arbitrary web page which can read your web based email conversations.

Cross Origin AJAX? Really?

For the reason stated above, a page can make an AJAX call to the same origin from which it originated. If I am allowed to coin an acronym stressing on the boundaries of AJAX, I would coin “AJAX-FOO”, which expands to “Asynchronous JavaScript And XML For Own Origin”. As soon as a new XMLHttpRequest is fired to a remote origin, browsers check the origin of the page with the destination of the request. If both are same, the call is allowed. Else, the call is blocked with an appropriate error message. So there is nothing like Cross Origin AJAX.

Understanding JSONP (TL;DR: It’s all about script tag hack!)

As they say, necessity is the mother of invention. When web2.0 APIs were introduced, they desperately wanted cross origin interactions. JSNOP was discovered as a hack/work-around to bypass the restrictions of Same Origin Policy.

The idea behind it is very simple. Same Origin Policy doesn't apply for scripts (and a couple of other elements too). A <script> tag in a web page can load JavaScript from any origin (i.e., when you embed jQuery.js pointing to a CDN, loading from a remote origin is allowed). Using this loophole, one can create cross origin requests.

Simple example to create your own JSONP service

1. Create a HTML page having two JavaScript files. In script1.js, create a function “processData”

function processData(data){ 
console.log('Hello '+data.firstName+' '+data.lastName);
}

2. In script2.js execute the above function by passing valid JSON data:

processData({firstName:'Krishna', lastName:'Chaitanya'}); 

3. When you load the page, both the script files load, code in the second file executes the function defined in the first file. This is an expected behavior.

4. Create a file “service.abc” (yes, create it with this dummy extension. This is going to be your web service) and place it in the same folder. Open it and write the same code as in step 2. Now open your web server (IIS or your preferable one), go to your site, open mime types section and add a new mime type “.abc” having the mime type value “text/javascript”.

5. Now remove reference to “script2.js” and add a reference to this new file “service.abc” in the head section like this:

<script type="text/javascript" src="service.abc"></script>

6. When you load the page now, you get the same behavior as that of script2. So far, everything is in the same origin. Place the file “service.abc” in another origin (simply create another website on a different port number-recollect that different ports means different origins) and reference it in script tag and the code still works.

What you have done is, you have loaded content from a remote service via script tag injection. This is the essence of JSONP. The idea of having a random file format “.abc” is just to show that any file which can serve script content will hold good for this. You may use your “.aspx”, “.asmx”, “.ashx” or whatever to achieve this.

Hence, JSONP is always a script Injection and has nothing to do with XMLHttpRequest object and AJAX.

How JavaScript libraries like jQuery help (mislead) you

If you use libraries like jQuery, they give you a common syntax which works for AJAX as well as JSONP hack. They do a lot of work behind the screens to make a JSONP script injection

$.getJSON('http://graph.facebook.com/zuck?callback=?', function (data) { 
console.log(data);
}

In the above API for Facebook, if the value for callback is given by the developer as “https://graph.facebook.com/zuck?callback=fetch”, facebook returns json data by wrapping it in the function “fetch” (open the link in your browser and check the output. Note: IE will ask to save the response as ".js" file.). If the function name is omitted, jQuery handles it in an interesting/tricky manner. It takes the success callback as the function to be executed (similar to “processData” function as declared above), creates a random function name and assigns the callback to it. The server too responds by wrapping its json data in the random function name which it got from the request (see the first screenshot in this blog post). Once the http transaction is done, jQuery destroys the random function.

(Note: To test the trick jQuery uses, I used burp proxy to intercept and pause the request sent by jQuery. While pausing, I typed jQuery’s random function name in browser’s console and it printed the definition of the function. After the response is received, I did the same and I got that function is undefined. This way I was able to deduce the trick jQuery uses for JSONP).

In this process, jQuery does not fire an AJAX call. All it does is injection of script tag and serving javascript in its response. Since the syntax for AJAX and JSONP are maintained the same, web developers tend to confuse about JSONP.

So what mime-type should be served for a successful JSONP request?

Well, this is a topic of confusion, at least for me. Since the served content is JavaScript, the preferred mime-type should be “application/javascript” or “text/javascript” or may be "application/json". In my demo, I’ve changed the mime type of the above service to “image/gif”, “text/css” etc and the script still worked in all modern browsers without any warnings. Also, there are cases where browsers show a "save file" dialog when wrong mime type is served. Enabling adhoc mime types has security concerns and research is being done in this area for standardizing mime-type. At least for now, “application/javascript” can be used and anyways CORS is the future, so no more content type worries.

Hope the article provided useful info. Share your thoughts or discuss if you see the need for any corrections. Happy coding Smile

Browser Internals: Content Isolation with Same Origin Policy-Microsoft UG Dev Day

Microsoft User Group Hyderabad (MUGH) has organized Developer Day at Broadridge Financial Solutions, Hyderabad this weekend. It was a half day event with very good line up of sessions and I had the opportunity to present on a very exciting topic-"Content Isolation with Same Origin Policy".

"Same Origin Policy (SOP)" is one of the foundations of web security, which is built into web browsers. Web developers often do not understand this policy clearly and work with several misconceptions. The goal of this session is to show how important SOP is for the web, how it is bypassed using hacks and what HTML5 offers as a standard to overcome its limitations. Getting a full understanding of SOP isn't easy in a one hour session as it is relatively vast and complex. However, I have tried to simplify several ideas and put them in one place in the slides. Folks who couldn't attend the session will also be benefitted from the slides.

Along with my talk, there was an interesting talk "One Service, Any Device, Any Platform-Web API" by fellow MVP Shravan and "A Lap around the new Windows Azure" by our super techie Phani, cofounder of BrainScale. It is really motivating to see close to 100 techies coming to learn cutting edge stuff over a weekend. That was a great time spent! See you in the next tech event. Happy coding :)

What web devs should know about HTTP ”Referer” header

Every HTTP request has a set of Request Headers which carry pieces of useful information from the client to the server. One such request header is the "Referer" header, which contains address of the previous page from which the current page was requested.

E.g., If you search for "HTML5" on google and click on the first result (link to wikipedia's page), you would be navigated to Wikipedia's HTML5 page and the "Referer" header contains the address of the previous page (i.e., google's search results page). Check the details in the below screenshot of IE9 F12 toolbar.

image

Over the years, “Referer” header (actual spelling should be “Referrer”, but it was misspelt in specs itself :p) has been used in several useful scenarios.

Fun with referrer:

By using “document.referrer” property in JavaScript, the address stored in referer header can be read. Using this web pages of Web 1.0 era displayed welcome messages, special offers, redirected to personalized landing pages etc.

if (document.referrer != ''){
alert('Hey! Welcome from '+ document.referrer);
}

CSRF protection:

Cross Site Request Forgery (CSRF) is a well known web based attack using which an attacker can make requests on behalf of the user. Leveraging CSRF, an attacker can construct GET/POST requests in a web page and make the victim open the page.

<!-- If this image tag is injected, it generates the below dangerous GET request -->
<img src="http://bank.com/funds/transfer?amount=10000&targetAccount=9876543210"/>

To defend against CSRF, the server has to differentiate between HTTP requests originating from a genuine user’s page vs an attacker’s page. Protecting against CSRF is a well explored area and it has several defenses such as using secret validation tokens, custom headers, Referer header etc. In most cases, the Referer header is used to check if the request is from the expected domain and not from attacker’s domain.

However, security experts have shown that referer headers can be easily stripped (Kotowicz’s demo) in all browsers and hence majority of CSRF defenses depending on referer header will fail.

Privacy Concerns:

In the era of social networks and personalization, data has become the currency of the web. By looking at the referer header, advertisements can learn from which page a user has visited the current page and provide more relevant ads. This means the browsing habits of users are being exposed to the cloud (Watch this Defcon video- How our Browser history is leaking into the cloud).

Till recently, Facebook exposed user’s unique Id in Referer header which caused serious concerns. Sites which are too concerned about privacy prefer to strip referer header and stay safe.

Damn! What web developers frequently use in their requirements is in fact not a recommended practice! Solutions are coming up!

Origin Header:

Researchers at Stanford Web security lab proposed that a new header called Origin Header should be used to uniquely identify requests. It is different from Referer header in that it just contains the origin (scheme://host: port) and not the entire address of the previous page. So this removes the privacy concern and can be used as protection against CSRF.

As far as I’ve seen, Origin header is implemented in Firefox, Chrome as an experimental feature and needs standardization (needs further verification).

Noreferrer: HTML5 introduces a new link type attribute called “noreferrer”. When an anchor tag is decorated with “rel=noreferrer” attribute, the pages which follow the hyperlink will not include referrer information in the header. This would pull down the privacy problem caused by Referer header. As of now, no browser supports this ‘noreferrer’ attribute.

So, the take away is, HTTP Referer header may be a handy option but it bears its own security and privacy problems and hence should be evaluated carefully. Instead, Origin header would be an ideal solution which would cater to the needs of web developers, respecting security and privacy.

Frame navigation policies in web browsers | One big reason why you should get rid of old browsers

Whether you are aware or not, frames are commonly used in most of the websites we use, for various purposes such as widgets in mashups, containers for advertisements, at the least for loading arbitrary documents into web pages. To serve this purpose <iframe> is used, while <frameset> and <frame> which were initially used for navigation are made obsolete in HTML5.

Frames are used primarily to isolate untrusted content such as remote scripts of widgets/ads etc., from interacting with rest of the DOM. Frames comply with Same Origin Policy if they load remote pages. This means, if an iframe is loaded with a page from same domain, it allows DOM manipulations to and from its parent page. Where as if it is loaded with a page from a different domain, it will restrict DOM manipulations and provides an isolated environment. The below code snippets should make this clear.

<!-- This is allowed -->
<iframe src="sameDomainPage.html"> </iframe>
alert(frames[0].contentDocument.body);  //works fine
 
<!-- This is **NOT** allowed -->
<iframe src="http://google.com"> </iframe>
alert(frames[0].contentDocument.body);  //throws error

As a developer, our knowledge of using frames ends here. We are happy with the secure isolation of content and don’t care beyond. But if we look a little deep into how browsers decide what should go into a frame, the scenario becomes scary!

Frame Navigation-who decides what:

Though the same origin policy of browsers isolates frame content of different domains, it has nothing to do with navigation of frames. i.e., if you know the id of a frame, you can navigate it to a different URL. (The browser window can be considered as a top level frame, having a visible address bar which iframes lack).

//Open google.com/ig and execute this script in console
myFrame=document.getElementsByTagName('iframe')[2];
myFrame.src="http://google.com"

Did you see the problems here? So if your trusted components are in iframes, what if someone can redirect the frame to a malicious URL? Which iframe has the permission to navigate which  other iframe on a complex mashup page like iGoogle? Even worse, can iframes on a different window/tab navigate the iframes on your page? What about popups? Do you feel the sense of insecurity now?

Well, the fact is, none of these are under the control of web developers. The policies used by browser vendors answer the above questions and developers should know these policies to understand how secure their websites are. Below are the navigation policies used by browsers. These are quite complex to have a deep understanding, so I just mention an outline :

1. Permissive policy: This policy prevailed in all browsers prior to 1999. It simply states that a frame can navigate any other frame. The frames can be in the same window or in different windows. This dangerous policy gave rise to what are called “Cross window attacks”. Several sites like CitiBank kept their password fields in iframes for secure sandboxing. So, an advertisement in another window or a popup containing malicious script can redirect the password frame to an evil frame  and grab credentials! IE6, Safari 3 and Flash by default used permissive policy.

2.  Window policy: Starting 2001, browsers implemented  a new policy called window policy, in order to prevent the cross window attacks. As per this, a frame can navigate frames in its own window and hence external popups/ads cannot affect frames of a different window. However, this gave rise to new line of attacks called “Same window attacks”. Using this policy, advertisements in the same window or evil gadgets (widgets)  in mashups (e.g., iGoogle) can redirect legitimate gadgets to malicious URLs. So Gadget hijacking became the fashion of the day due to this policy. Firefox 2 and Opera 9 followed this policy which is dangerous to work with today’s gadget rich Web 2.0 applications.

3. Child Policy: This is a stricter policy, which says that a frame can navigate only its direct child. IE6 team wanted to enable this by default but did not since majority of the existing sites are not compatible with this. Such a strict implementation will prevent even the navigation of legitimate frames which are from same origin as the parent. So this policy could not be implemented.

4. Descendant policy: This is another stricter policy which was designed by IE7 team and it provides the best trade-off between security and compatibility. As per this, a frame can navigate only its descendants. So this prevents the cross window as well as same window attacks. HTML5 working group has standardized this policy

Frame navigation policies in browsersThe above picture explains the policies better. The outer blue box is the browser frame, the inner blocks are iframes and the arrows indicate who can navigate the content of what. Notice that child policy is a subset of descendant., descendant is a subset of window., window is a subset of permissive policies.

All modern browsers (IE8+, FF3+, Safari 3+, Opera >9) follow the descendant policy to navigate frames and hence are more secure than their older  counterparts. This is one of the several hundred reasons why you should get rid of older browsers when you use modern web 2.0 applications.

While we discussed about frame navigation so far, the next problem is how do legitimate frames communicate with legitimate content. e.g., communication between partner gadgets in a mashup. Though there are hacks like using fragment identifier or APIs like HTML5 postMessage, attacks like Reply attack, Recursive mashups attack are made possible with a good understanding of these policies. I shall cover on these and few more interesting scenarios in my upcoming posts e.g., what happens if your website can be framed by attackers and leverage descendant policy for backdoor communication?

Reference: The Stanford security researchers have published an excellent research paper-"Secure Frame Communications in Browsers (pdf)", which is the motivation for this post. Due credits to the authors. Suggest you to go through it for better picture.

Happy secure coding :)

ECMAScript 5 'Strict mode' support in browsers. What does this mean?

If you are closely following how the web is shaping up lately, you would be thrilled, like me, looking at pace at which things are changing. There are hundreds of things happening on the new HTML5, CSS3, JavaScript front, frequently releasing browser versions and contest for speed, supporting modern standards etc. One such innovations is ECMAScript 5 Strict Mode support in modern browsers. Recently, IE10 platform preview 1 announced 'strict mode' support in Mix 2011 conference . The goal of this article is to explain what 'Strict Mode' support in modern browsers mean and how developers can take advantage of it.

 

I am aware of JavaScript. But what is ECMAScript 5?

Similar to W3C which standardizes HTML specifications, ECMA is the body which standardizes JavaScript. ECMA-262 is the specification which defines the JavaScript language, interchangeably referred as Jscript/ECMAScript. The JavaScript code which we web developers have been using since a decade is as per the guidelines defined in the Third edition ofECMAScript (ES3 in short) released in December,1999.

 

ES3 was released in a hurry and it had several loopholes, which means that the future versions have to be built on top of these loopholes to support older pages (I strongly recommend reading grandpa Crockford's book, JavaScript- The Good Parts to know the good and bad parts of the language). ECMAScript 4 had several ambitious, drastic changes in its specs but was never released due to conflicts regarding backward compatibility, fearing a broken web. So,ECMAScript 5 (ES5 in short) was released in December 2009, more as a transitional version of the specification, adding few cool features, without much changes in the syntax (Check the ECMAScript specs archive).

 

Few new features in ES5:

ECMAScript 5 includes few new APIs, computational methods for things which can already be done in ES3. One of the most interesting features is the newer object model in ES5, using which, newer objects can be created easily and their extensibility can be controlled. Similar to higher programming languages like Java or C#, JavaScript objects now have getters and setters, non-enumerable methods, sealed and frozen objects  etc. ES5 also describes native JSON support, which includes methods like JSON.parse() and JSON.stringify() which remove dependency on external libraries for working with JSON.

 

Each browser has its specific implementation of ES5, as per its JS engine. ES5 support in IE 9 is different from ES5 support in FF 4.

 

ES5 Strict mode:

'Strict mode' is an innovative feature of ES5. It is a restrictive variant of ES5 implementation, which follows a slightly different semantics from the normal JavaScript. Someof the deprecated, bad parts of JavaScript in ES3 specification are disabled or thrown as exceptions in 'strict mode'. It is not backward compatible with older browsers and has to be opted-in by the developer.

To enable ES5 in strict mode, just use the string "use strict" as the first line of your JavaScript code. This will make the entire program to run in strict mode.

 

Alternately, you can place "use strict" in a function, to restrict only to that context.

 

function isStrictMode(){

return !this;

} 

//returns false, since 'this' refers to global object and '!this' becomes false

 

function isStrictMode(){   

"use strict";

return !this;

} 

//returns true, since in strict mode, the keyword 'this' does not refer to global object, unlike traditional JS. So here,'this' is null and '!this' becomes true.

 

I hope to cover detailed demos on this in the following posts.

 

The good parts of  ES5 Strict Mode:

In JavaScript, if a variable is not explicitly declared, it is implicitly assigned global scope.This may look fun to code in JS, but is a nightmare to maintain in huge projects. Strict mode throws error if variables are assigned values without declaring and hence implicit global variables will not be created. This is a huge win for the language as well as for huge projects.

 

Few such good changes in strict mode are: Assignment to a read-only property will throw error, object literals having 2 or more properties with same name will throw error, functions having 2 arguments with same name will throw error etc. All these changes will help in making the code more reliable, readable and manageable.

 

The IE team launched  a 'Strict Mode' demo in their IE Test Drive HTLM5 demos. You can check if your browser supports ES5 Strict mode or not by checking the "Try Strict Mode" demo. Download IE10 platform preview 1 if you haven't and start playing with it!

 

P.S: Happy to share that Microsoft has recognized my community contributions and awarded me "Microsoft MVP award" for the second consecutive year. In 2010, it was under ASP.NET/jQuery category and now for 2011, it is under "InternetExplorer" category. Thanks to Microsoft for the encouragement Smile

Why web devs should get excited about IE9!

A more beautiful web was launched few days back! After several platform previews, beta build, release candidate build, community feedback and rigorous testing, the final (RTM) version of Internet Explorer-9 has hit the market. On the first day of its release (Mar 14, 2011) itself IE9 saw 2.35 million downloads, which is 27 downloads/second! If you aren't one among them, you should download IE9 RIGHT NOW from here: www.beautyoftheweb.com (Yes, it is an official, trusted site by Microsoft).

Before going any further, let me confess something. I'm a web developer enjoying modern web technologies. I use IE, Chrome, Firefox, Opera, Safari interchangeably. I used hacks to support older browsers. I want HTML5, CSS/JS support, good debugging tools, cross browser compatibility. I want the web to win, like many of you and hence want to share my excitement about IE9, which I call as a win for the web :)

If you are a newbie to IE9, you can check the top features of IE9. Since there are several resources on the features, I want to concentrate on why web devs should get excited about IE9.

As a web dev, whether you love it or hate it, you need to support all browsers. In most of the cases, IT enterprises and their clients are Microsoft shops and will prefer IE+Windows over any other combination. You may be a Firebug geek (check IE9 developer tools), but your client doesn’t bother about it. He needs his site to work flawlessly in IE primarily and of course support the rest. (This is a well known fact which does not need proofs). With rest of the browsers updating quickly, IE was a little behind the race w.r.t supporting newer web standards. "Better late than never", as the saying goes, IE9 comes as a great boon by supporting most of the modern web standards, easing developer's life. So even if you are a firefox/chrome power user, being a web dev, you need to respect IE9.  I'm sure you would love it if the 'critic' mask is removed.  Now comes the below question.

Is IE9 the most 'modern' browser on earth w.r.t web standards?

The answer to this debatable question depends on your definition of the term 'modern'. If implementing web standards drafted by W3C just a couple of minutes back is what you call 'modern', then IE9 is NOT.  If providing 'site-ready' HTML5/CSS3/JS support which developers and consumers can rely without fear is what you call 'modern', then yes, IE9 is the most modern browser :)

Did you know?

Microsoft started something called HTML5 Labs, where it prototypes early, unstable web standards and features proposed by standards bodies like W3C (Web Sockets, IndexedDB etc.). It offers prototypal implementations for unfinished parts of HTML5 which interested developers can try, but consumers can’t yet rely on.

By clearly separating prototype implementations from main stream browser product, many negative consequences can be avoided.  Implementing latest standards in a rush and rolling them back (e.g., Chrome H.264, Google gears etc) for whatever reason is only a pain for web devs. IE team knows what it's competitors already have but is waiting for stability of standards, so that developers can have peaceful time.

What's hot now?

IE9 now supports WebM video format via WebM components for IE9. You should check this article for further details, where Dean Hachamovitch, Corporate Vice President of Internet Explorer, makes a truly valid point by saying "The people who build and use the Web deserve practical and consistent video support rather than ideology."

Resources from my friends, which you should check:

  * Article on few FAQs (myths?) about IE9  by  Harish Ranganathan, Microsoft India

  * Article on History of Internet Explorer from IE to IE9 by Kunal Chowdhury, Microsoft MVP

So, it's time for web devs to rejoice and play with modern (and stable) web standards, widening creative and technical horizons. Keep checking the official IE blog for more updates. Enjoy the beauty of the web :)

NOTEMicrosoft TechEd 2011 is just a few days away. If you are unable to make to it, you may watch TechEd 2011 LIVE and watch the sessions, industry trends for FREE. Dont Miss It!! I'm attending in person. See you there! :)