Server-sent event (SSE) with ASHX (.Net framework 4.6)

It took me a while to solve the Server-sent event (SSE) implementation in .Net framework 4.6.

Notes:

- This new ASHX should not access any session data. Otherwise, the user access will be "blocked". To solve this, you might have to store the user session data in a database or in memory (using static Dictionary or something like that).

- If the user open multiple tabs, the server must force closing the earlier connections. This will reduce the use of server & browser resources.

- Creating more EventSource might have side effect in the browser as it will reduce the performance in downloading the CSS, images, etc. If you need more EventSource object, you may have to create more sub-domains such as WWW1, WWW2,...

- Don't forget that any request to ASP.NET will consume a shared thread which might affect C# performance. The server connection has a limit too.

- It's best to use HTTP/2 for SSE which has better performance as compared to HTTP/1.

Here's the server side coding - add a new handler - notify_me_endlessly.ashx.

<%@ WebHandler Language="C#" Class="notify_me_endlessly" %>

using System;
using System.Web;
using System.Threading;
using System.Threading.Tasks;

public class notify_me_endlessly : IHttpHandler
{

    /// <summary>
    /// NOTES:
    /// - use HTTP/2 (with SSL/TLS) to reduce the request payload.

    /// Otherwise, the network might be flooded.
    ///
    /// </summary>
    /// <param name="context"></param>
    public void ProcessRequest(HttpContext context)
    {
        // to indicate the content type is for EventSource() .
        context.Response.ContentType = "text/event-stream";

        // hold the connection in a separate thread.
        // NOTES: this might overload the server with client connections!!

        Task t = Task.Run(() =>
        {

            // run endlessly until the client close the connection      //<<=======
            while (true)
            {

                try
                {
                    //this is IMPORTANT. The message

                    //sent to the caller must be "data: xxx".
                    context.Response.Write(string.Format("data: {0}\n\n",
                    DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

                    System.Diagnostics.Debug.WriteLine(string.Format("{0}-sending data to browser..",
                        DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

                    context.Response.Flush();

                    Thread.Sleep(2000);
                }
                catch //(Exception x)
                {
                    System.Diagnostics.Debug.WriteLine(string.Format("{0}-failed sending data to client",
                          DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));
                    break;
                }
            }

        });

        // wait until end of the TASK.           //<<=======
        t.Wait();

        // disconnect the caller - this is optional and the connection

        // will be closed upon exiting this proc.       
        //context.Response.Close();

        System.Diagnostics.Debug.WriteLine(string.Format("{0}-client has quit",
                                                   DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

    }

    public bool IsReusable
    {
        get
        {
            return false;
        }
    }

}

Here's the front end which consumes the SSE:

<!DOCTYPE html>
<html>
<head>
    <title></title>
    <meta charset="utf-8" />
</head>
<body>

    <h1>
        Msg from server
    </h1>

    <input type="button" id="btnListen" value="Start Listening" onclick="startListening(); return false;" />
    <input type="button" id="btnListen" value="Stop Listening" onclick="stopListening(); return false;" />
    <ul id="my-list"></ul>


    <script type="text/javascript">
        var sse = null;
        var mylist = document.getElementById('my-list');

        function startListening() {

            //-------------------------------
            var newElement = document.createElement("li");
            newElement.textContent = "message: connecting..";
            mylist.appendChild(newElement);

            //reset before use.
            if (sse != null) {
                sse.close();
                sse = null;
            }

            sse = new EventSource('notify_me_endlessly.ashx');
          

            //-------------------------------
            sse.addEventListener("open", function (event) {
                var newElement = document.createElement("li");
                newElement.textContent = "message: connected";
                mylist.appendChild(newElement);

                console.log('onopen=', event);

            }, false);

            sse.addEventListener("error", function (event) {
                var newElement = document.createElement("li");

                if (sse.readyState == 2) {
                    newElement.textContent = "ERR message: connection closed";
                }
                else if (sse.readyState == 1) {
                    newElement.textContent = "ERR message: open";
                }
                else if (sse.readyState == 0) {
                    newElement.textContent = "ERR message: connecting..";
                }
                else {
                    newElement.textContent = "ERR message: failed-";
                }

                mylist.appendChild(newElement);

                //22.Jun.20,lhw-you might have to call sse.close() manually upon any error
                // and reconnect after x seconds using setInterval() - in this case, you
                // will have full control on the reconnecting behavior (since all
                // browser has slightly different implementation).

                console.log('onerror=', event);

            }, false);


            sse.addEventListener("message", function (e) {
                var newElement = document.createElement("li");
                newElement.textContent = "message: " + e.data;
                mylist.appendChild(newElement);

                console.log('onmessage=', event);

            }, false);

        }

        function stopListening() {

            // stop listening to the server message.
            if (sse) {

                // this proc inform the browser to stop reconnecting the server
                // in case of connection failure.
                sse.close();
                sse = null;

                var newElement = document.createElement("li");
                newElement.textContent = "message: stopped by user";
                mylist.appendChild(newElement);
               
            }

        }

    </script>

</body>
</html>

Parse from string - is it JObject or JArray?

While we are developing Web API, we might expect the caller to pass in either an object or array of object. Of course it would be best to handle array of object as the API paramter. But, we are not always have the luck to change the API parameter type since the existing system is connected by many third party apps.

Basically, the existing code is handling the incoming parameter that is string type, a JSON formatted value. We are parse it correctly if we know that type:

  using Newtonsoft.Json.Linq;

  string s = @"{'name': 'MIKE', 'age':'23'}";
  JObject jobj = JObject.Parse(s);

  string s2 = @"[{'name': 'MIKE', 'age':'23'}, {'name': 'MICKEY', 'age':'25'}]";
  JArray jarr = JArray.Parse(s2);

Let's say, we modified our Web API to handle array of object while the existing code is only handling object, it will crash:

  try
  {
      // let's crash it
      JObject jobj2 = JObject.Parse(s2);
  }
  catch (Exception x)
  {
      Console.WriteLine(x.Message);
  }

In this case, you have to parse the incoming text into JToken.

  JToken jt = JToken.Parse(s);
  JToken jt2 = JToken.Parse(s2);

To identify whether it is JObject or JArray:

  bool b = jt is JObject;
  bool b2 = jt is JArray;

Removing some HTTP response headers

In the web.config of ASP.NET website project:

1. Add the following line to remove the X-AspNet-Version header.

<system.web>
  <httpRuntime enableVersionHeader="false"/>
</system.web>


2.  Removing X-Powered-By header.

  <system.webServer>
    <httpProtocol>
      <customHeaders>
        <remove name="X-Powered-By" />
      </customHeaders>
    </httpProtocol>   
  </system.webServer>

3. Removing Server header which you may do it in the global.asax

void Application_BeginRequest(object sender, EventArgs e)
{
    var application = sender as HttpApplication;
    if (application != null && application.Context != null)
    {
        application.Context.Response.Headers.Remove("Server");
    }
}

To change the default cookie name in ASP.Net

<sessionState cookieName="mySessID" />

HttpWebRequest and automatic decompression

Continue from our previous post "Posting compressed data in JSON format to the ASP.NET website using WinForm", in HttpWebRequest, there is a property called "AutomaticDecompression" (boolean) which is able to decompress the server response without extra codes.

Now, the question is that if we set this property to true, does it compress the data before sending it to the web server? After research and confirm that it does not compress the data before sending.

Below is the sample code to upload the data after

            string url = "http://localhost:30000/myHandler.ashx";
            System.Net.HttpWebRequest req = System.Net.WebRequest.Create(url) as System.Net.HttpWebRequest;
            req.ContentType = "text/json";
            req.Method = "POST";
            req.AutomaticDecompression = System.Net.DecompressionMethods.GZip
                                        | System.Net.DecompressionMethods.Deflate;

            // generate dummy data.
            string s = "";
            for (int i = 0; i < 100; i++)
            {
                s += Guid.NewGuid().ToString();
            }

            using (System.IO.StreamWriter w = new System.IO.StreamWriter(req.GetRequestStream()))
            {
                w.WriteLine("helo me.." + s);
            }

In the ASHX handler, you may verify the content length that has been submitted from the client:

public class myHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {      
        System.Diagnostics.Debug.WriteLine(string.Format("{0}-header=>"
                + context.Request.ContentLength.ToString(),
                                            DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

        context.Response.ContentType = "text/plain";
        context.Response.Write("Hello World");
    }

    public bool IsReusable
    {
        get
        {
            return false;
        }
    }
}

         

Posting compressed data in JSON format to the ASP.NET website using WinForm

We have posted an article on how to post JSON data from WinForm to ASP.NET/ASHX in 2015 (as shown below). In the earlier article, it does not optimize the payload size and reduce the communication time.

    http://laucsharp.blogspot.my/2015/07/posting-data-in-json-format-to-aspnet.html

In order to speed up the communication between the WinForm client and ASP.NET/ASHX, we must reduce the payload size.To do that, the contents must be compressed before we upload the data and decompress upon receiving it.

There are 2 compression algorithms for HTTP/HTTPS communication: gzip and deflat.

Here is the code to compress the contents with GZip algorithm. You need this code in both WinForm and ASP.NET/ASHX.

        public static void GZip(Stream stream, string data)
        {
            byte[] b = System.Text.Encoding.UTF8.GetBytes(data);
            GZip(stream, b);
        }

        public static void GZip(Stream stream, byte[] data)
        {
            using (var zipStream = new GZipStream(stream, CompressionMode.Compress, true))
            {
                zipStream.Write(data, 0, data.Length);
            }
        }
To decompress the GZip contents. You need this code in both WinForm and ASP.NET/ASHX.

        public static string GUnZipToString(Stream stream)
        {
            byte[] b = GUnZip(stream);
            return System.Text.Encoding.UTF8.GetString(b);
        }

        public static byte[] GUnZip(Stream stream)
        {
            using (var zipStream = new GZipStream(stream, CompressionMode.Decompress, true))
            using (MemoryStream ms = new MemoryStream())
            {
                zipStream.CopyTo(ms);
                return ms.ToArray();
            }
        }
In the WinForm app, we used to write the JSON data directly to the HttpWebRequest like this:

        // write the json data into the request stream.
        using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
        {
                writer.Write(json_data);
        }

In order to reduce the payload, we should write the compressed data by doing this:

        // to indicate we accept gzip content.
        request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip");

        // to indicate that the content is compressed using gzip algorithm.
        request.Headers.Add("Content-Encoding", "gzip");

        GZip(request.GetRequestStream(), json_data);

 In the ASP.NET/ASHX, we used to read the contents like this:

        // get the contents from the request stream
        Stream stream = context.Request.InputStream;
        using (StreamReader r = new StreamReader(stream))
        {
            s = r.ReadToEnd();
        }

Now, we have to read the contents and then decompress it. "Content-Encoding" is the indicator whether the contents is in GZip, deflat or not compress.

        if (HelperFunc.IsGZipContent(context))
        {
                 // read the compressed content.
                s = GUnZipToString(context.Request.InputStream);          
        }
         else
        {
                // get the contents from the request stream
                Stream stream = context.Request.InputStream;
                using (StreamReader r = new StreamReader(stream))
                {
                        s = r.ReadToEnd();
                }
        }

Here is the function to check whether the contents is compressed in GZip format. Please take note that the sample codes in this article did not support "deflat" algorithm.

        public static bool IsGZipContent(System.Web.HttpContext ctx)
        {
            string enc = ctx.Request.Headers["Content-Encoding"];

            if (string.IsNullOrEmpty(enc))
            {
                return false;
            }

            return  enc.ToLower().Contains("gzip");
        }

You will find more information about the static and dynamic compression in IIS:

   https://docs.microsoft.com/en-us/iis/configuration/system.webserver/httpcompression/

To find out why we need to compress the data, please refers to the following link:

   https://developer.mozilla.org/en-US/docs/Web/HTTP/Compression

To check if the web server is able to handle TLS1.2

You can find lots of information on how to enable the TLS 1.2 in Windows/IIS server. Somehow, their article left out the steps on how to verify if the TLS 1.2 really has been enabled.

Here is the step:

1. Make sure that you have install openssl utility.

2. Run the following command in command prompt.

      cd\
      cd OpenSSL-Win64\bin
      openssl s_client -connect google.com:443 -servername google.com -tls1_2

If TLS1.2 has been enabled, you will see the SSL client cert and session ID appear in the response.

You may test to see if any other TLS version has been enabled or disable.

• -tls1
• -tls1_1
• -ssl3

To enabled TLS1.2 in Windows server run, save the following text into a text file. The file extension is ".reg". After that, double click on this file to merge the registry.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2]

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client]
"DisabledByDefault"=dword:00000000
"Enabled"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server]
"Enabled"=dword:00000001

Enforcing TLS1.2

TLS 1.2 has been in born since 2008 and yet many of us did not know about it until new requirement came to us...

To enforce the TLS 1.2 in a website, you need to add a Global.asax and then add the following line:

    void Application_Start(object sender, EventArgs e)
    {
        System.Net.ServicePointManager.SecurityProtocol = (System.Net.SecurityProtocolType)3072;
    }

Once the above code has been added, all HTTP request will be served in TLS 1.2 that includes the web service (ASMX) and HTTP request call to external website. And make sure that the project is compiled under .Net 4.6 and everything will be ok.

Notes: your program will work with TLS1.2 only. An exception will be thrown if the server does not enabled TLS 1.2.

There is a catch... if some other app is consuming the resource on your website, they must be on TLS 1.2 as well. Otherwise they will fail to make any connection to your website.

HTTP to HTTPS auto redirect

To enforce all client connections to use HTTPS, you need to redirect the HTTP connection to HTTPS. This can be done easily in IIS.

1) Install URL Rewrite for IIS which can be downloaded from the following URL:

     https://www.iis.net/downloads/microsoft/url-rewrite


2) Add the following settings that to the web.config file (withing WebServer section):

<rewrite>
    <rules>
        <rule name="HTTP to HTTPS" patternSyntax="Wildcard" stopProcessing="true">
            <match url="*" />
            <conditions logicalGrouping="MatchAny">
                    <add input="{HTTPS}" pattern="off" />
            </conditions>
            <action type="Redirect" url="https://{HTTP_HOST}{REQUEST_URI}" appendQueryString="false" redirectType="Permanent" />
        </rule>
    </rules>
</rewrite> 

To learn more about the rewrite component,

    https://docs.microsoft.com/en-us/iis/extensions/url-rewrite-module/url-rewrite-module-configuration-reference

Forcing CSS or JS file to reload after website update

"The CSS or JS (JavaScript) file did not download by the browser after the website has been updated" - I guess most of the website developers faced this problem before and this problem can be resolved easily in ASP.Net.

The cause of the problem: usually, we include a CSS file into the website using the following code and the file name is "static". With the static file name, the browser will use the cache version of it during the next visit to the website and this is the cause of the problem. Sometimes, pressing F5 might not reload because the URL is the same and then the cache version will be reused.

    <link href="/css/myStyle.css" rel="stylesheet" />

To resolve this issue, you need to need to declare a new page class that inherits from System.Web.UI.Page OR declare an extension method (you need to figure out how to do this). Then, all your web page will have to inherit from this new page class.

Basically, the solution is to append the file modification date and time to the end of the CSS or JS file as shown below. Once you have done that, the browser will be force to load from the web server automatically without pressing F5.

   <link href="/css/ui.css?20171205164334" rel="stylesheet"/>
   <script src="/js/myScript.js?20171211150501"></script>


The codes that do the magic:

        public string AttachCss(string css)
        {
            if (string.IsNullOrEmpty(css))
            {
                return string.Empty;
            }

            string[] s = css.Replace(";", ",").Split(',');
            string s2 = string.Empty;
            string f, f0;
            FileInfo fi;

            foreach (var item in s)
            {
                // default path is "/css"
                f0 = "~/css/" + item;
                f = this.Server.MapPath(f0);

                if (File.Exists(f))
                {
                    fi = new FileInfo(f);
                    f = this.ResolveUrl(f0) + "?" + fi.LastWriteTime.ToString("yyyyMMddHHmmss");

                    s2 += string.Format("<link href=\"{0}\" rel=\"stylesheet\"/>", f);
                }
            }

            return s2;
        }

        public string AttachJs(string js)
        {
            if (string.IsNullOrEmpty(js))
            {
                return string.Empty;
            }

            string[] s = js.Replace(";", ",").Split(',');
            string s2 = string.Empty;
            string f, f0;
            FileInfo fi;

            foreach (var item in s)
            {
                // default path is "/js"
                f0 = "~/js/" + item;
                f = this.Server.MapPath(f0);

                if (File.Exists(f))
                {
                    fi = new FileInfo(f);
                    f = this.ResolveUrl(f0) + "?" + fi.LastWriteTime.ToString("yyyyMMddHHmmss");
                    s2 += string.Format("<script src=\"{0}\"></script>", f);
                }
            }

            return s2;
        }
      
To use the code, you need to call AttachCss() and AttachJs() within the ASPX file. You may include multiple files into the call.

<asp:Content ID="Content2" ContentPlaceHolderID="ContentPlaceHolder1" runat="Server">

    <%= this.AttachCss("myStyle.css;myStyle2.css") %>
    <%= this.AttachJs("myScript.css;myScript2.css") %>   
   
   (your HTML goes here...)
</asp:Content>

Cross-Origin Request Blocked (CORS)

To speed up the development and future upgrade, we split the huge application into multiple AJAX services. Each AJAX service in running in it's own application pool and it can be run on different server. The design works perfectly. But, when you want to consume the AJAX services through the browser, you bang your head: "Cross-Origin Request Blocked".

This is the error message that appeared in the Firefox:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost/schedule/q?code=tx&ts=1507099862873. (Reason: CORS header ‘Access-Control-Allow-Origin’ does not match ‘(null)’).

Google Chrome returned an error message that is slightly different:

Failed to load http://localhost/schedule/q?code=tx&ts=1507099946004: The 'Access-Control-Allow-Origin' header contains multiple values '*, *', but only one is allowed. Origin 'http://localhost:56269' is therefore not allowed access.

Now, if you are googling for the solution, you will end up with add the following settings in the web.config.

  <httpProtocol>
    <customHeaders>
      <add name="Access-Control-Allow-Origin" value="*" />
      <add name="Access-Control-Allow-Headers" value="Content-Type" />
      <add name="Access-Control-Allow-Methods" value="GET, POST, PUT, DELETE, OPTIONS" />
    </customHeaders>
  </httpProtocol>

But, the wild card origin is no longer supported. You ended up with adding the specific origin.

  <httpProtocol>
    <customHeaders>
      <add name="Access-Control-Allow-Origin" value="http://localhost:56292" />
      <add name="Access-Control-Allow-Headers" value="Content-Type" />
      <add name="Access-Control-Allow-Methods" value="GET, POST, PUT, DELETE, OPTIONS" />
    </customHeaders>
  </httpProtocol>

Imagine that you are hosting your AJAX services in multiple servers with different sub-domains.... the above solution will not work. This is because you are not allowed adding more than one domain name to "Access-Control-Allow-Origin".

To solve the problem, we need to handle the OPTIONS verb by adding the following settings in the web.config:

  <system.webServer>
    <handlers>
      <add verb="OPTIONS" name="check_opt" path="*" type="ajaxLib.CORS_OPTIONS" />
    </handlers>
  </system.webServer>

And below is the simplified code that allows CORS:

namespace ajaxLib {
public class CORS_OPTIONS : IHttpHandler
{
  public void ProcessRequest(HttpContext context)
  {
        if (context.Request.HttpMethod.ToUpper() == "OPTIONS") {
           string s = context.Request.Headers["Origin"];

           if (!string.IsNullOrWhiteSpace(s))
           {
             context.Response.AppendHeader("Access-Control-Allow-Origin", s);
             context.Response.AppendHeader("Access-Control-Allow-Headers", "Content-Type");
             context.Response.AppendHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
           }
    }
  }

  public bool IsReusable { get { return false; }}

}}

Two possibilities if you want to use the above code in the live environment,

1. If your service allows public access without any restriction, skip checking the Origin value.
2. If your service allows specific domain to access, you must check the Origin value before return it to the caller.

Decoupling the code with "notification" design

In C#, there are many different ways to decouple the program and the most common ways of doing so is to use one of the following:

• event - it's multicasting. You listen to what you need.
• interface  - it's dynamic. You can incorporate the interface into any of your class.
• callback using Action<T> or Func<T> - you handle what you are interested.

Other than the above, there is another way to decouple the code - using "notification messages". For example, the lowest level WndProc (in WinForm) which processes all Windows messages. Let's make use of this strategy in our C# program.

The core of this strategy - the publisher & subscriber + multi-threaded object. Let's call it NotificationServer. This core object allows the publisher to send the notification messages into the message pool. It also allows the subscriber to subscribe and listen to the notification messages that they are interested. You will find tons of example on how to implement the publisher & subscriber + multi-threaded in the Internet.


The notification message object contains the following properties:

• message ID - the ID that allows the subscriber to identify the purpose of it.
• session ID - it's GUID type and it's used in conjunction with broadcast flag.
• data - it's an object to be passing around.
• broadcast - it's a flag which tells the NotificationServer whether it should send the message to a specific subscriber or all subscribers. This can be very useful if you are implementing a TCP NotificationServer.
• should feedback - it's a flag that indicate whether it waits for the NotificationServer's respond. This can be very useful if you are implementing a TCP NotificationServer.

The NotificationServer design

• Embedded NotificationServer- the implementation is to have a publisher & subscriber + multi-threaded object.
• The fun does not stop here - you can embed the NotificationServer class into a TCP server class and all communications are done through TCP communication. In this case, you will have a TCP NotificationServer which is able to run as a Windows Service. The publisher and subscribers can be any program, ASP.Net web page or another Windows Service. The publisher and subscribers could be running from the same computer or different computer.  

In our TCP NotificationServer implementation, the notification message is serialized into JSON format. We chose JSON format because we are reusing the TCP NotificationServer in various projects.

A better way to configure Intelli-sense for Javascript in VS2015

Now, I'm moving toward the direction of developing fat client using Javascript (JS) and I need a
better way of enable the intelli-sense support for Javascript in the fat client.

Earlier, I found out that we can include the reference in each JS file by adding the following line.
When the number of JS files that you are developing are increasing, added this reference line might be headache.

   /// <reference path="jquery-1.7.js" />

Here is a better way:

1. Add a "_references.js" file into "js" folder (let says you are keeping all your JS files in a folder call "js" instead of "scripts").

2. In the Quick Launch bar (located at top right corner), key in "Javascript" and look for "IntelliSense -> References"  option. Clicks on it.



3. Choose "Implicit (Web)" option.
4. In the "Add a reference to current group" field, key in "~/js/_references.js".
5. Click Add button.
6. Click OK.



7. Back to the Solution Explorer. Right click on _references.js and you will find two new options.

- Auto-sync- means that VS will add all the JS files automatically.
- Update Javascript References - this is manually sync the JS files.