Server-sent event (SSE) with ASHX (.Net framework 4.6)

It took me a while to solve the Server-sent event (SSE) implementation in .Net framework 4.6.

Notes:

- This new ASHX should not access any session data. Otherwise, the user access will be "blocked". To solve this, you might have to store the user session data in a database or in memory (using static Dictionary or something like that).

- If the user open multiple tabs, the server must force closing the earlier connections. This will reduce the use of server & browser resources.

- Creating more EventSource might have side effect in the browser as it will reduce the performance in downloading the CSS, images, etc. If you need more EventSource object, you may have to create more sub-domains such as WWW1, WWW2,...

- Don't forget that any request to ASP.NET will consume a shared thread which might affect C# performance. The server connection has a limit too.

- It's best to use HTTP/2 for SSE which has better performance as compared to HTTP/1.

Here's the server side coding - add a new handler - notify_me_endlessly.ashx.

<%@ WebHandler Language="C#" Class="notify_me_endlessly" %>

using System;
using System.Web;
using System.Threading;
using System.Threading.Tasks;

public class notify_me_endlessly : IHttpHandler
{

    /// <summary>
    /// NOTES:
    /// - use HTTP/2 (with SSL/TLS) to reduce the request payload.

    /// Otherwise, the network might be flooded.
    ///
    /// </summary>
    /// <param name="context"></param>
    public void ProcessRequest(HttpContext context)
    {
        // to indicate the content type is for EventSource() .
        context.Response.ContentType = "text/event-stream";

        // hold the connection in a separate thread.
        // NOTES: this might overload the server with client connections!!

        Task t = Task.Run(() =>
        {

            // run endlessly until the client close the connection      //<<=======
            while (true)
            {

                try
                {
                    //this is IMPORTANT. The message

                    //sent to the caller must be "data: xxx".
                    context.Response.Write(string.Format("data: {0}\n\n",
                    DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

                    System.Diagnostics.Debug.WriteLine(string.Format("{0}-sending data to browser..",
                        DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

                    context.Response.Flush();

                    Thread.Sleep(2000);
                }
                catch //(Exception x)
                {
                    System.Diagnostics.Debug.WriteLine(string.Format("{0}-failed sending data to client",
                          DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));
                    break;
                }
            }

        });

        // wait until end of the TASK.           //<<=======
        t.Wait();

        // disconnect the caller - this is optional and the connection

        // will be closed upon exiting this proc.       
        //context.Response.Close();

        System.Diagnostics.Debug.WriteLine(string.Format("{0}-client has quit",
                                                   DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

    }

    public bool IsReusable
    {
        get
        {
            return false;
        }
    }

}

Here's the front end which consumes the SSE:

<!DOCTYPE html>
<html>
<head>
    <title></title>
    <meta charset="utf-8" />
</head>
<body>

    <h1>
        Msg from server
    </h1>

    <input type="button" id="btnListen" value="Start Listening" onclick="startListening(); return false;" />
    <input type="button" id="btnListen" value="Stop Listening" onclick="stopListening(); return false;" />
    <ul id="my-list"></ul>


    <script type="text/javascript">
        var sse = null;
        var mylist = document.getElementById('my-list');

        function startListening() {

            //-------------------------------
            var newElement = document.createElement("li");
            newElement.textContent = "message: connecting..";
            mylist.appendChild(newElement);

            //reset before use.
            if (sse != null) {
                sse.close();
                sse = null;
            }

            sse = new EventSource('notify_me_endlessly.ashx');
          

            //-------------------------------
            sse.addEventListener("open", function (event) {
                var newElement = document.createElement("li");
                newElement.textContent = "message: connected";
                mylist.appendChild(newElement);

                console.log('onopen=', event);

            }, false);

            sse.addEventListener("error", function (event) {
                var newElement = document.createElement("li");

                if (sse.readyState == 2) {
                    newElement.textContent = "ERR message: connection closed";
                }
                else if (sse.readyState == 1) {
                    newElement.textContent = "ERR message: open";
                }
                else if (sse.readyState == 0) {
                    newElement.textContent = "ERR message: connecting..";
                }
                else {
                    newElement.textContent = "ERR message: failed-";
                }

                mylist.appendChild(newElement);

                //22.Jun.20,lhw-you might have to call sse.close() manually upon any error
                // and reconnect after x seconds using setInterval() - in this case, you
                // will have full control on the reconnecting behavior (since all
                // browser has slightly different implementation).

                console.log('onerror=', event);

            }, false);


            sse.addEventListener("message", function (e) {
                var newElement = document.createElement("li");
                newElement.textContent = "message: " + e.data;
                mylist.appendChild(newElement);

                console.log('onmessage=', event);

            }, false);

        }

        function stopListening() {

            // stop listening to the server message.
            if (sse) {

                // this proc inform the browser to stop reconnecting the server
                // in case of connection failure.
                sse.close();
                sse = null;

                var newElement = document.createElement("li");
                newElement.textContent = "message: stopped by user";
                mylist.appendChild(newElement);
               
            }

        }

    </script>

</body>
</html>

HttpWebRequest and automatic decompression

Continue from our previous post "Posting compressed data in JSON format to the ASP.NET website using WinForm", in HttpWebRequest, there is a property called "AutomaticDecompression" (boolean) which is able to decompress the server response without extra codes.

Now, the question is that if we set this property to true, does it compress the data before sending it to the web server? After research and confirm that it does not compress the data before sending.

Below is the sample code to upload the data after

            string url = "http://localhost:30000/myHandler.ashx";
            System.Net.HttpWebRequest req = System.Net.WebRequest.Create(url) as System.Net.HttpWebRequest;
            req.ContentType = "text/json";
            req.Method = "POST";
            req.AutomaticDecompression = System.Net.DecompressionMethods.GZip
                                        | System.Net.DecompressionMethods.Deflate;

            // generate dummy data.
            string s = "";
            for (int i = 0; i < 100; i++)
            {
                s += Guid.NewGuid().ToString();
            }

            using (System.IO.StreamWriter w = new System.IO.StreamWriter(req.GetRequestStream()))
            {
                w.WriteLine("helo me.." + s);
            }

In the ASHX handler, you may verify the content length that has been submitted from the client:

public class myHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {      
        System.Diagnostics.Debug.WriteLine(string.Format("{0}-header=>"
                + context.Request.ContentLength.ToString(),
                                            DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

        context.Response.ContentType = "text/plain";
        context.Response.Write("Hello World");
    }

    public bool IsReusable
    {
        get
        {
            return false;
        }
    }
}

         

Posting compressed data in JSON format to the ASP.NET website using WinForm

We have posted an article on how to post JSON data from WinForm to ASP.NET/ASHX in 2015 (as shown below). In the earlier article, it does not optimize the payload size and reduce the communication time.

    http://laucsharp.blogspot.my/2015/07/posting-data-in-json-format-to-aspnet.html

In order to speed up the communication between the WinForm client and ASP.NET/ASHX, we must reduce the payload size.To do that, the contents must be compressed before we upload the data and decompress upon receiving it.

There are 2 compression algorithms for HTTP/HTTPS communication: gzip and deflat.

Here is the code to compress the contents with GZip algorithm. You need this code in both WinForm and ASP.NET/ASHX.

        public static void GZip(Stream stream, string data)
        {
            byte[] b = System.Text.Encoding.UTF8.GetBytes(data);
            GZip(stream, b);
        }

        public static void GZip(Stream stream, byte[] data)
        {
            using (var zipStream = new GZipStream(stream, CompressionMode.Compress, true))
            {
                zipStream.Write(data, 0, data.Length);
            }
        }
To decompress the GZip contents. You need this code in both WinForm and ASP.NET/ASHX.

        public static string GUnZipToString(Stream stream)
        {
            byte[] b = GUnZip(stream);
            return System.Text.Encoding.UTF8.GetString(b);
        }

        public static byte[] GUnZip(Stream stream)
        {
            using (var zipStream = new GZipStream(stream, CompressionMode.Decompress, true))
            using (MemoryStream ms = new MemoryStream())
            {
                zipStream.CopyTo(ms);
                return ms.ToArray();
            }
        }
In the WinForm app, we used to write the JSON data directly to the HttpWebRequest like this:

        // write the json data into the request stream.
        using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
        {
                writer.Write(json_data);
        }

In order to reduce the payload, we should write the compressed data by doing this:

        // to indicate we accept gzip content.
        request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip");

        // to indicate that the content is compressed using gzip algorithm.
        request.Headers.Add("Content-Encoding", "gzip");

        GZip(request.GetRequestStream(), json_data);

 In the ASP.NET/ASHX, we used to read the contents like this:

        // get the contents from the request stream
        Stream stream = context.Request.InputStream;
        using (StreamReader r = new StreamReader(stream))
        {
            s = r.ReadToEnd();
        }

Now, we have to read the contents and then decompress it. "Content-Encoding" is the indicator whether the contents is in GZip, deflat or not compress.

        if (HelperFunc.IsGZipContent(context))
        {
                 // read the compressed content.
                s = GUnZipToString(context.Request.InputStream);          
        }
         else
        {
                // get the contents from the request stream
                Stream stream = context.Request.InputStream;
                using (StreamReader r = new StreamReader(stream))
                {
                        s = r.ReadToEnd();
                }
        }

Here is the function to check whether the contents is compressed in GZip format. Please take note that the sample codes in this article did not support "deflat" algorithm.

        public static bool IsGZipContent(System.Web.HttpContext ctx)
        {
            string enc = ctx.Request.Headers["Content-Encoding"];

            if (string.IsNullOrEmpty(enc))
            {
                return false;
            }

            return  enc.ToLower().Contains("gzip");
        }

You will find more information about the static and dynamic compression in IIS:

   https://docs.microsoft.com/en-us/iis/configuration/system.webserver/httpcompression/

To find out why we need to compress the data, please refers to the following link:

   https://developer.mozilla.org/en-US/docs/Web/HTTP/Compression

Using partial HTML UI + ASHX to speed up page loading

Just to share with everyone that I developed many LOB app (line of business application) and I'm still developing new LOB for my clients. LOB is very different from blog engine, corporate websites and static websites. In LOB, we can happily ignore the contents to be "readable" by SEO because some of the contents were loaded by AJAX.

As per my last blog dated 26th-June-2016, I mentioned the new strategy: JQuery + AJAX + partial HTML UI. Now, the question is how is partial HTML UI that can help out is speeding up the page loading? The answer is simple, we need to rely on the browser cache by checking the "If-Modified-Since" flag in the request header. Then, responding either status code 304 (resource has not been modified) or returning the partial HTML to the browser.

You will find tons of references if you are searching for "asp.net If-Modified-Since". Below is one of the reference that I found:

  http://madskristensen.net/post/use-if-modified-since-header-in-aspnet

The down side of this strategy is that the user will feel a bit slower on the first request to load the full page. But, the subsequent page loading or if the user is requesting for the same page, then, the time taken will be shorter. For example, we want to develop a page for user to key in the sales invoice and it allows the user from choosing the item from the list. The sales invoice is stored in a HTML file (not ASPX) and the item list HTML design is stored in another HTML file (where this item list will be reused by supplier invoice).

One of the advantage using this strategy is that it allows all these partial HTML file to be hosted in CDN (Content Delivery Network). Then, the whole LOB app will be loaded faster than using only one ASPX which could be crazily huge and hard to reuse some of the HTML design.

Note: "partial HTML UI" can be refer as "template" and it does not contains the HEAD and BODY tags. It just contains some DIV-s which eases the web designer to design and test in the browser. You don't need a programmer to start full coding but just some simple JQuery and AJAX to complete the demo.

System design with ASHX web services (JQuery + AJAX + partial HTML UI)

Recently, we are working on a few new systems with ASP.NET. In the past, we are using Page (ASPX) + ScriptManager and we are facing some limitation in the system design which includes the following:

• The system does not allow the user to add a new "item" into the drop down list on the fly.
• The drop down list contains a few hundred items and we need to incorporate the 'search' functionality or paging to avoid all items to be loaded in one shot.
• The data submitted to server failed to meet the validation process and causing the entire page sent back to the client (which travel from client to the server and then back to the client).

To solve these types of problem, we need to have a new design to the foundation of the system.

• We must use JQuery + AJAX + partial HTML UI design so that it allows the user from adding "item" to the drop down list on the fly. The partial HTML UI will appear on the screen as a popup for user adding new item. After the user has submitted the new item to the server, validated OK and it will be added to the drop down list on the fly (with JQuery) without reloading the page or navigating to another page.
• The drop down list that contains lots of item will be replace by a textbox. Upon the user clicking on this textbox, the items will be loaded from the server (with AJAX calls) and then display in the popup (with JQuery + partial HTML UI). To improve the user's experience, you may consider the auto-complete feature upon the user typing the text or divide the data using the paging concept.
• We must do more AJAX calls for submitting the user input to be validated by the system. In case of any failure such as validation failed, the server should returns the error message only. This avoids the entire page to be re-created in the server and then send to the browser.

With the new design, the system becomes more responsive and lesser network traffic. But, we still have a problem on how to handle the AJAX call. Are we going to have one ASHX to handle one process (that will end up on lots of ASHX)? Or are we going to have only one ASHX entry point that handles all the requests?

To solve this problem, here is the list of frequent use "web service" to be implemented with Handler (ASHX):

• ~/q  - this web service handles the "query" that includes CRUD (create, return, update & delete) processes, daily process, ad-hoc process and all other business processes. The report request is another area which you may consider to put into this service.
• ~/t - this web service returns the "HTML template" (partial HTML UI design) to be injected to the current web page. By creating the partial HTML UI file, it allows the designer to work on the layout without have to go through all the JQuery + the DOM element generation (i.e., low level stuffs). Modifying the DOM elements using JQquery is very time consuming and it requires a more expensive Javascript programmer. But, we have done it with a cheaper costing. The nice partial HTML UI has been done by the designer and the programmer requires to populate the JSON data into the appropriate placeholder.
• ~/f - this web service handles all the file services that include upload, download/view. For example, when the user calls out the "contact list", it shows the profile photo of the contact. This profile photo IMG SRC is "~/f?contact_id=124567" where "contact_id" is the primary key value of the contact. It does not point to any physical file name. The "f" service will do all the necessary at the server side and returns the binary of the photo (an image file).

To setup the above shortcut, you have to create mapped URL in web.config. For example,

  <system.web>
    <urlMappings>
      <add url="~/q" mappedUrl="~/myWebService/q.ashx"/>
    </urlMappings>
  </system.web>

The design these web services:

• Client is making a request to the web service:
• "code" - the command code, object type or process code to be executed.
• "action" - this includes CRUD and other actions (such as run daily job, run hour job).
• "query parameters" - the query parameters are wrapped into a JSON object. For example, the client is requesting the customers who is owing more than $10,000 for more than 90 days.
• Responding to the client:
• "msg" - the message to the client. "ok" to indicate the query has successfully executed. Otherwise, it contains the error message.
• "list" - the list of JSON formatted data requested by the client. This information is optional.

Some of you might be thinking why we are not using REST for these web services. The answer is simple: we don't need to given definition to the "method". Such as PUT or POST method. What if the user wants to execute an ad-hoc process (POST or custom method)?

Routing to ASHX

Here is the piece of code that I found in CodeProject.com. By adding this extention method, you will be able to route the request to ASHX:

namespace System.Web.Routing
{
    public class HttpHandlerRoute : IRouteHandler
    {
        private String _virtualPath = null;
        private IHttpHandler _handler = null;

        public HttpHandlerRoute(String virtualPath)
        {
            _virtualPath = virtualPath;
        }

        public HttpHandlerRoute(IHttpHandler handler)
        {
            _handler = handler;
        }

        public IHttpHandler GetHttpHandler(RequestContext requestContext)
        {
            IHttpHandler result;
            if (_handler == null)
            {
                result = (IHttpHandler)System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(_virtualPath, typeof(IHttpHandler));
            }
            else
            {
                result = _handler;
            }
            return result;
        }
    }

    public static class RoutingExtensions
    {
        public static void MapHttpHandlerRoute(this RouteCollection routes, string routeName, string routeUrl, string physicalFile, RouteValueDictionary defaults = null, RouteValueDictionary constraints = null)
        {
            var route = new Route(routeUrl, defaults, constraints, new HttpHandlerRoute(physicalFile));
            RouteTable.Routes.Add(routeName, route);
        }

        public static void MapHttpHandlerRoute(this RouteCollection routes, string routeName, string routeUrl, IHttpHandler handler, RouteValueDictionary defaults = null, RouteValueDictionary constraints = null)
        {
            var route = new Route(routeUrl, defaults, constraints, new HttpHandlerRoute(handler));
            RouteTable.Routes.Add(routeName, route);
        }
    }
}

To access the routing data in ASHX, you need to do this:


            var o = context.Request.RequestContext.RouteData.Values["id"];
            if (o != null)
            {
                q = o.ToString();
            }

Reference:
http://www.codeproject.com/Tips/272258/ASP-net-HttpHandler-Routing-Support

Web Service (asmx) vs Handler (ashx)

In ASP.NET environment, it comes with quite extensive way of handling the web request. There is Page class (aspx), Handler (ashx), Web service (asmx), Web service in WCF (asmx), etc. It's all upto the developer to decide the best way to write their program.

Let's compare the Web Service (asmx) and Handler (ashx):

• With Web Service you can run your Visual Studio and add the web service as web reference. This will create the necessary classes for you. With web service, the serialization and deserialization of the data object will be handle by .Net framework. This simply means that you can work with Web Service without having deep knowledge in the "background" process.
• With Handler you must find out what is the parameter to be pass to the handler. No code generator that is able to help out over here. It provides very basic features to serve the request and you will have to serialize/deserialize your data objects manually (you may use JavaScriptSerializer to deal with the JSON data). It does not limit you from returning the data in JSON, XML or CSV format. It's all depend on your design.

With Web Service, it saves you from lots of codes in the serialization process. With Handler, it allows to develop low level request & response. Both has it's own pros and cons.

There is no conclusion on which one is better. It goes back to you to choose which one is suitable for the project that you work on.

Session is not missing in the Handler

I still remember the first time that I'm developing with generic handler (.ashx) and found out that the session is missing. This is because the handler implements only the basic feature (as compared to Page class).

In order for the AJAX call accessing the session information, your handler must implement the following interface:

    System.Web.SessionState.IRequiresSessionState

OR

    System.Web.SessionState.IReadOnlySessionState

That's all you need.

Improving the web app by using AJAX + Handler

The following diagram explains the way to improve the web application response and reduces the server load by using AJAX call to the Handler.

For example, the "default.aspx" allows the user to post message and view all the messages. In the default.aspx, you will not use the GridView control to show all the messages. Instead, you make AJAX requests to the LoadMessage.ashx and show the result in the default.aspx. In this way, no viewstate and no ASP controls will be generated by LoadMessage.ashx.

The LoadMessage.ashx may return the result in 2 ways:

• formatted result (with HTML/TABLE tags or whatsoever) OR
• result in JSON format. Of course, the result in JSON format requies lesser bandwidth. But, you need to write some Javascript to format the result and show it in the default.aspx.

Another function provided by PostMessage.ashx will be a lot simpler. When the user type their message in the default.aspx (at the client site), you just need to retrieve the message from the screen and then make AJAX calls to the PostMessage.ashx (with the appropriate parameters).

Handler (ashx) VS Page (aspx)

We used to do development with Page (aspx) which comes with lots of rapid application development feature such as postback handling, drag & drop support, etc. But, this becomes expensive when the application is hosted in the Internet (not in a LAN). The viewstate, the control object instance generation, etc are all come with a price. A more complicated situation is that when you try to use AJAX (ScriptManager + UpdatePanel), the performance become questionable (when traffic goes up) and the coding become very complicated.

On the other hand, Handler (ashx) is a class that provides very basic functionality as compared to Page. There is no visual designer for Handler. No viewstate. No control object generation. No postback. You have to handle everything by yourself. The advantage of Handler as compared to Page is the performance (by giving up all the rapid application development features).

With JQuery and JSON, implementing AJAX become very easy. Modifying the DOM objects at the client site become easier as well. Using JQuery AJAX call to the Handler will boost up the performance because the viewstate is no longer require to transmit back and fore between the server and the client. At the server side, it also does not requires to re-generate all the control object instances.

Try this out:

• Design a data entry form in Page (aspx).
• Click on the Submit button and the client will make a AJAX call to the Handler (ashx) with the user input values.
• Then, the Handler will return the result in JSON format to the client.
• The client will then check the result and make the appropriate response.

 Benefits of using Handler (ashx) + Page (aspx):

• Faster response time
• Reduce the data to be transmitted between the server and the client.
• The server CPU load will reduce due to the Page call is lesser.

 Challenges:

• The developer requires to more JavaScript/JQuery knowledge.
• Harder to debug.
• Because the input controls in the Page is sometimes unpredictable (in .Net 3.5), getting the input field with JQuery will become troublesome.

Hosting images using Handler (ASHX)

In HTML, when we want to display images, we are using the following tag:

   <img src="images/myProfilePhoto.jpg"/>

That's very simple and has nothing much to complaint. But, you won't be able to trace who has requested the image file especially when the privacy issue comes into place... it becomes very nasty. In this case, we need to use ASP.NET Handler (ASHX) to host the images.

Advantage of hosting images through Handler:

• You can hide the image file path. For example, your web app is located in D drive and all image files are stored in E drive (where you don't have to expose the image folder in IIS).
• You may track down the visitor information that includes their IP address, browser, etc.
• You may stored the images in a database and host it through the Handler - you don't need to create an image file physically and then write the phyicsal file name into "SRC" attribute. You just need to set the SRC attribute to your ASHX handler (refers to the sample code in the link below).
• You may cache the images in the server.

In the HTML, you will have something like this:

  <img src="showPhoto.ashx?memberID=0001"/>

Where "showPhoto.ashx" is the Handler that you develop and "memberID" is the query parameter. Your handler should return the profile photo (in byte array) for member ID 0001.

For the code on how to implement the Handler or ASHX:

  http://stackoverflow.com/questions/994135/image-from-httphandler-wont-cache-in-browser
  http://stackoverflow.com/questions/1507572/streaming-databased-images-using-httphandler

For full explanations on the Handler:

   http://support.microsoft.com/kb/308001