HttpWebRequest and automatic decompression

Continue from our previous post "Posting compressed data in JSON format to the ASP.NET website using WinForm", in HttpWebRequest, there is a property called "AutomaticDecompression" (boolean) which is able to decompress the server response without extra codes.

Now, the question is that if we set this property to true, does it compress the data before sending it to the web server? After research and confirm that it does not compress the data before sending.

Below is the sample code to upload the data after

            string url = "http://localhost:30000/myHandler.ashx";
            System.Net.HttpWebRequest req = System.Net.WebRequest.Create(url) as System.Net.HttpWebRequest;
            req.ContentType = "text/json";
            req.Method = "POST";
            req.AutomaticDecompression = System.Net.DecompressionMethods.GZip
                                        | System.Net.DecompressionMethods.Deflate;

            // generate dummy data.
            string s = "";
            for (int i = 0; i < 100; i++)
            {
                s += Guid.NewGuid().ToString();
            }

            using (System.IO.StreamWriter w = new System.IO.StreamWriter(req.GetRequestStream()))
            {
                w.WriteLine("helo me.." + s);
            }

In the ASHX handler, you may verify the content length that has been submitted from the client:

public class myHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {      
        System.Diagnostics.Debug.WriteLine(string.Format("{0}-header=>"
                + context.Request.ContentLength.ToString(),
                                            DateTime.Now.ToString("d/MMM/yy @ HH:mm:ss.fff")));

        context.Response.ContentType = "text/plain";
        context.Response.Write("Hello World");
    }

    public bool IsReusable
    {
        get
        {
            return false;
        }
    }
}

         

Decoupling the code with "notification" design

In C#, there are many different ways to decouple the program and the most common ways of doing so is to use one of the following:

• event - it's multicasting. You listen to what you need.
• interface  - it's dynamic. You can incorporate the interface into any of your class.
• callback using Action<T> or Func<T> - you handle what you are interested.

Other than the above, there is another way to decouple the code - using "notification messages". For example, the lowest level WndProc (in WinForm) which processes all Windows messages. Let's make use of this strategy in our C# program.

The core of this strategy - the publisher & subscriber + multi-threaded object. Let's call it NotificationServer. This core object allows the publisher to send the notification messages into the message pool. It also allows the subscriber to subscribe and listen to the notification messages that they are interested. You will find tons of example on how to implement the publisher & subscriber + multi-threaded in the Internet.


The notification message object contains the following properties:

• message ID - the ID that allows the subscriber to identify the purpose of it.
• session ID - it's GUID type and it's used in conjunction with broadcast flag.
• data - it's an object to be passing around.
• broadcast - it's a flag which tells the NotificationServer whether it should send the message to a specific subscriber or all subscribers. This can be very useful if you are implementing a TCP NotificationServer.
• should feedback - it's a flag that indicate whether it waits for the NotificationServer's respond. This can be very useful if you are implementing a TCP NotificationServer.

The NotificationServer design

• Embedded NotificationServer- the implementation is to have a publisher & subscriber + multi-threaded object.
• The fun does not stop here - you can embed the NotificationServer class into a TCP server class and all communications are done through TCP communication. In this case, you will have a TCP NotificationServer which is able to run as a Windows Service. The publisher and subscribers can be any program, ASP.Net web page or another Windows Service. The publisher and subscribers could be running from the same computer or different computer.  

In our TCP NotificationServer implementation, the notification message is serialized into JSON format. We chose JSON format because we are reusing the TCP NotificationServer in various projects.

Queue processing design

Queue concept is straight forward - append new item at the tail and dequeue from the head. It does not tell you how to process the requests in the queue.

You can enqueue new requests with multiple threads and processing the requests with 1 worker thread or multiple threads. Then, here comes with the processing designs:

Sequential (synchronous) processing the requests

With only 1 worker thread processing the requests in the queue, this is going to be slower as compared to the multiple worker threads but it has advantages. The advantage of sequential processing is when it comes to file I/O operation or network I/O operation. Just imagine that the exit has one door and only one person is able to pass through at one time.

Parallel (asynchronous) processing the requests

Thanks to multi-core and multi-CPU. With more than 1 worker thread processing the requests, it reduces the client wait time (on the response) and increases the server throughput.

Multi-thread has nothing to do with the "reducing the processing time for a request"! The "processing time" is depending on your codes and the how much the codes can be optimized (in the development time and also run time).

If there is a long run request and the request is handling by a worker thread (in a thread-pool), then, that worker thread is blocked. It won't be able to handle other request until it finished the current request. And if you have many of long running requests, then, all threads in the thread pool might not be able to reduce the response time. The server is now consider as "busy" even though the client is able to keep sending their requests into the queue.

Can we have unlimited threads in the pool..?!

The answer is no. If you have instantiated too many threads without using thread pool, then, the memory will be depleted... and the program will crash. 

What happen to the requests in the queue if my program crash?

All requests will be disappear from the memory. Your program may be designed with auto-restart in case it crashed. But, you won't be able to recover the requests which was stored in the memory.

In case you need something better than storing the requests in memory..

1. Each thread in the thread pool must maintain individual processing statistics and these statistics will be updating to the database hourly. With this information, the system administrator will be able to tell how many worker threads were blocked, in which time zone and which request blocks it. Then, he will be able to justify whether to upgrade or replace the server.

The statistics to be keep track by each thread in the thread pool should include:

- Number of requests that has been processed
- Total processed time (ms)
- Highest processed time (ms)
- Request name for highest processed time

- Peak hour - 24 time zone in a day (just get the current hour value will do)

2. Using database as queue storage instead of memory - the main advantage is that the server will be able continue from where it's left before crashing. With this approach, there will be some overhead in storing the requests into database. But, those overhead is justifiable with the crash-proof design.

Another advantage of storing the requests into database is that the secondary server will be kick-in if all the worker threads in the thread pool (in the primary server) are blocked. The secondary server will be notify by the primary server (through WCF or socket). Then, the secondary server will query the database for the pending requests and process it accordingly.

Two ways to handle the respond to the client

- The result will be stored in the database and then notify the primary server. Then, the primary server will query the result and responds to the client.

- OR the result will be send to the primary server (through WCF or socket) and then responds to the client.

Queuing the data/request

Non-blocking... hey just call me whenever the result is ready.. I don't want to wait here.

Whenever we talk about blocking (synchronous) or non-blocking (asynchronous), obviously it is related to the multi-threading and queue.

Thread is created to execute some codes without blocking the main thread (using the time slice of a CPU or run the thread in a separate CPU core). This improves the responsiveness of the program (i.e., the main thread).

In the server program, it needs to serve many clients concurrently. So, the server program have to queue the requests and let the client go (i.e., without asking them to wait for the result). Queue is always first in first out (i.e., FIFO).

• The worker thread will always process the request that come first. The result will be send to the client through callback (please refers to the Push Design article earlier).
• On the other hand, the client request  will be appended at the end of the queue. And the client will wait for the result through callback.

With this processing order, all requests will be served based on its' "request time".

Things will become more complicated with the following design:

• Thread pool (i.e, there are multiple worker threads that handle the request) - you can find many open source C# thread pool libraries which smartly create more threads when there are many requests and reduces the number of threads when the number of requests reduce. Some will even create threads based on the number of CPU core.
• Request can be prioritized - with prioritization, it allows the urgent request to jump queue even though it came in late. You can imagine that the server program has multiple queues (one for each priority) and the highest priority will have threads to standby to serve the urgent request.

OK. Let's continue the previous Push Design topic.

You need "command queue" and "callback data queue"

With WCF, the socket programming becomes easier. But, without the "non-blocking" design in mind, the communication process will make the server or the client unresponsive. The unresponsiveness will be severe when the number of concurrent clients increase and the amount of data to be transmit become larger. To alleviate this problem, you need to implement thread pool and queue into both server program and client program.

In the client program, you need this:

• Command queue - when the user click "submit request to the server", the command (written in WCF) should go into a "command queue" (this queue is residing at the client site). Then, one worker thread will send this request to the server. Since we are designing "non-blocking" program, the worker thread should not wait for response from the server. It will continue to send the next request/command to the server until the queue is empty. From the user experience, the user will feel that clicking the submit button does not freeze the screen (this is something good). For example, the user is using Internet browser to open multiple tabs and each tab is requesting different web pages.
• Callback data queue - once server completed and the request and sends the result back to the client (through callback), the client should store the result into a queue. This is the second queue that you need aside from the command queue. Upon receiving the response, the worker thread should dispatch the result to the respective "caller" (which could be a screen) until the queue is empty.

In the server program you need this:

• Command queue - when the client program sends a request, it will be appended to the queue (this queue is residing at the server site). The client should not wait for the result or else they could be blocking the server (this could end up with resource contention problem where multiple clients are competing for the same resource). A worker thread will pickup this request and do all the necessary process. Upon completion, it will append the result to the callback data queue and let another worker thread to dispatch the result to the client.
• Callback data queue - same as the client program, this is another queue aside from the command queue. The purpose of this queue is to let the command worker thread to process the rest of the requests immediately after one request has been completed. Making a callback from to the client might face latency problem (i.e, not really "realtime" but some unexpected network traffic out there). With a thread that only handles the callback, even though the network connection is slow it won't affect the command work thread (the processing time will be maintaining). Now, the callback worker thread will take it's own sweet time to send the result to the client. No worry about the process time. No worry about the limited command worker thread in the pool.

The command queue and callback data queue should work in conjunction with thread pool. You may have one thread pool to take care one queue or one thread pool that take cares all the queues.

Push design

Everything in the communication has time limit

In push design, one of the bigger challenge is to make sure that both server and client side are storing the data/request into a "queue" and then to be handle when some threads are free to process the data/request. By using the "queue", the client/server can end the current method call after the data/request has been queued for later processing.

In the WCF context, you have two types of design to process the data/request:

1. using "function" design (works like "DateTime.Now" which returns the value immediately) - for example, the client sends "current time" command to the server and expecting the server responding (almost) immediately at the end of the calls.

2. using "callback" design - for example, the client sends "current time" command to the server and does not wait for the server respond. Instead, the server will send the current time through callback.

Both designs have pros and cons and it all depends on your need.

- The "callback" design allows the server to take it's precious time to prepare the necessary data for the client. In case the server is busy or the resources were blocked, it just have to wait until those resources were freed up. It also allows the server to schedule the process later. Upon completion, the server will make callback to the client. This is acceptable if it is not a real-time system.

- The "function" design - the client is always waiting for the result and it needs it now. By using this design, your server is running on deadlock risk (i.e., competing for the resources and locked the resource that other client is asking for). Since all the clients want it now, the deadlock will occur as soon as the same resources were requested by multiple clients. Of course, the deadlock can be avoided with proper locking mechanism.

Even with WCF, the connection will get disconnected

This is not true if you have full control over the server and client. WCF allows the system administrator to tune the "keep alive" time limit. Just in case you don't have the full server access, you need to do something to keep the connection alive.


This can be done by sending NOOP command (i.e., a dummy command that does not perform any action) from client to the server - this will keep the connection alive. In case the connection has broken, you just need to re-establish it.

To send the NOOP command repeatedly, you just need a System.Threading.Timer object which queuing the NOOP command in every 1 or 2 seconds.

How to get the updated data - Push VS Pull

In a client and server environment or cloud environment, your program often requires to monitor the updates on the server. There are two ways to catch the updates: either using a Push or Pull design.

• Push design - with this design, when there is an updates happened, the server will notify the client program. The design will be more complicated (in both client program and server program) as compared to the Pull design.
• Pull design - with this design, the client program will continuously query the server for the updates. Of course, this design is very simple but it comes with a bigger costs (in terms of bandwidth and server processing power) when the number of connections grow.

In order to serve more client connections and reducing the bandwidth consumption, you will have to implement the Push design.

Using socket or web socket to implement the push design:

• This is one of the basic element to implement the push design. So, you must learn how to write socket program. With .Net, you may use WCF (Windows Communication Foundation) to implement this idea but you still need to learn the technical details of what is all about socket and how it works with different configuration.
• Imagine that user A key in a new blog post throught a website and then all the followers will be notified within a few seconds. In this case, the server will send a signal (either using TCP or UDP) to the "online users" (i.e., the user must run a client program and sitting down in the computer to wait for the incoming signal). The preferred way to send the signal is using UDP which you can find lots of information about TCP vs UDP.
• Other than how to send the signal, one of the challenge is the how secure is your data when it is travelling from the server to the client or vice versa. Of course, with WCF, you have the choice of choosing different configuration. In other platform (other than .Net), you will might have to implement the security over the socket communication using SSL/TLS. Just to share with you that you can implement SSL/TLS in Python easily.
• I guess we are quite lucky with the modern programming languages because most of the modern programming langauges able to support asynchronous design with a few keywords changes. We need to learn about async programming as well or otherwise the server program will not be able to scale-up.

Errors that appear in the WCF client app...

I was setting a new server to host my WCF app and I hit the wall with the following error messages. I spent 2 days in solving this problem. I guess, many people is wondering how to resolve these interesting errors.

• The server has rejected the client credentials.
• The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:59.9687500'.
• The communication object, System.ServiceModel.ServiceHost, cannot be used for communication because it is in the Faulted state.

I have developed a WCF solution in WinForm which allows you to comment out certain settings in the server and client so that you can reproduce the above mentioned error message.

  https://github.com/lauhw/WcfWinForm

Please beware that if you are running the WCF server and client in the same computer, the above error might not appear until you run them in a separate computer.

In a nutshell:

1. Both server and client config file must have the same settings in the "Binding" record. This includes the "security mode".
2. The "identity" settings must be the same in both server and client config file.
3. In the live environment, make sure you remove the "MEX" endpoint unless you want anyone to be able to query the interface.
4. Don't forget to open the port in the firewall.

In case you are hitting the same error mentioned above, the fastest way to solve this problem is to run the test WCF solution (as mentioned above). Make sure that the test solution works. After that, compare the differences in app.config between the test solution and your solution.

Hope you don't have to spend 2 days to solve the configuration problem. ;)

Get client endpoint in WCF

Below is the code to get the client endpoint in WCF:

public string GetClientEndpoint()
{
    MessageProperties properties = OperationContext.Current.IncomingMessageProperties;
    RemoteEndpointMessageProperty endpoint = properties[RemoteEndpointMessageProperty.Name]
                                as RemoteEndpointMessageProperty;
    string s= string.Format("Client endpoint => {0}:{1}", endpoint.Address, endpoint.Port);
    return s;
}

In case you are developing a custom transport channel, please refer to the following article:

http://blogs.msdn.com/b/phenning/archive/2007/08/08/remoteendpointmessageproperty-in-wcf-net-3-5.aspx

Scaling out your system using web services

There are many different ways to improve the system respond time: one of the strategy is call scaling out. In our system design, we are implementing the "basic modules" (i.e., the system infrastructure) in web service and it is able to achieve this easily.

Web service does not have a huge different from a website. The noticeable different is that it does not have user interface (or web pages) for visitor to access. You may imagine that the web service like a mobile phone station/transmitter which is providing connection to the mobile phone. In .Net, the web service is implemented in ASMX format, WCF (Windows Communication Foundation) or simply ASPX (which returns the data in XML or JSON format).

Now, the best part of web service is not something that will make our solution cool. Instead, it is able to scale out to a server other than the web server (that is hosting the website). In this case, our customer might end up with a web server to host the website, a few web servers that host the web services.

Many programmers argue that web service is slow because of XML SOAP involves in the communication. It's true that web service in ASMX is slow but the visitor won't really feel it because of these communications were made among the web servers which is sitting next to each other. Of course, the web service implemented in XML SOAP is not suitable for real-time application. The real-time application requires low level socket programming and the handling strategy will be different.

WCF TCP Port Sharing

Check this out. WCF allows you to share the same port among the services.

http://msdn.microsoft.com/en-us/library/ms734772.aspx
http://msdn.microsoft.com/en-us/library/ms731810.aspx

The article below has a diagram that depicts the port sharing:
http://www.codeproject.com/KB/WCF/Port_Sharing__in_WCF.aspx

Configuration:
http://blogs.msdn.com/b/drnick/archive/2006/04/14/configuring-http.aspx