Posting compressed data in JSON format to the ASP.NET website using WinForm

We have posted an article on how to post JSON data from WinForm to ASP.NET/ASHX in 2015 (as shown below). In the earlier article, it does not optimize the payload size and reduce the communication time.

    http://laucsharp.blogspot.my/2015/07/posting-data-in-json-format-to-aspnet.html

In order to speed up the communication between the WinForm client and ASP.NET/ASHX, we must reduce the payload size.To do that, the contents must be compressed before we upload the data and decompress upon receiving it.

There are 2 compression algorithms for HTTP/HTTPS communication: gzip and deflat.

Here is the code to compress the contents with GZip algorithm. You need this code in both WinForm and ASP.NET/ASHX.

        public static void GZip(Stream stream, string data)
        {
            byte[] b = System.Text.Encoding.UTF8.GetBytes(data);
            GZip(stream, b);
        }

        public static void GZip(Stream stream, byte[] data)
        {
            using (var zipStream = new GZipStream(stream, CompressionMode.Compress, true))
            {
                zipStream.Write(data, 0, data.Length);
            }
        }
To decompress the GZip contents. You need this code in both WinForm and ASP.NET/ASHX.

        public static string GUnZipToString(Stream stream)
        {
            byte[] b = GUnZip(stream);
            return System.Text.Encoding.UTF8.GetString(b);
        }

        public static byte[] GUnZip(Stream stream)
        {
            using (var zipStream = new GZipStream(stream, CompressionMode.Decompress, true))
            using (MemoryStream ms = new MemoryStream())
            {
                zipStream.CopyTo(ms);
                return ms.ToArray();
            }
        }
In the WinForm app, we used to write the JSON data directly to the HttpWebRequest like this:

        // write the json data into the request stream.
        using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
        {
                writer.Write(json_data);
        }

In order to reduce the payload, we should write the compressed data by doing this:

        // to indicate we accept gzip content.
        request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip");

        // to indicate that the content is compressed using gzip algorithm.
        request.Headers.Add("Content-Encoding", "gzip");

        GZip(request.GetRequestStream(), json_data);

 In the ASP.NET/ASHX, we used to read the contents like this:

        // get the contents from the request stream
        Stream stream = context.Request.InputStream;
        using (StreamReader r = new StreamReader(stream))
        {
            s = r.ReadToEnd();
        }

Now, we have to read the contents and then decompress it. "Content-Encoding" is the indicator whether the contents is in GZip, deflat or not compress.

        if (HelperFunc.IsGZipContent(context))
        {
                 // read the compressed content.
                s = GUnZipToString(context.Request.InputStream);          
        }
         else
        {
                // get the contents from the request stream
                Stream stream = context.Request.InputStream;
                using (StreamReader r = new StreamReader(stream))
                {
                        s = r.ReadToEnd();
                }
        }

Here is the function to check whether the contents is compressed in GZip format. Please take note that the sample codes in this article did not support "deflat" algorithm.

        public static bool IsGZipContent(System.Web.HttpContext ctx)
        {
            string enc = ctx.Request.Headers["Content-Encoding"];

            if (string.IsNullOrEmpty(enc))
            {
                return false;
            }

            return  enc.ToLower().Contains("gzip");
        }

You will find more information about the static and dynamic compression in IIS:

   https://docs.microsoft.com/en-us/iis/configuration/system.webserver/httpcompression/

To find out why we need to compress the data, please refers to the following link:

   https://developer.mozilla.org/en-US/docs/Web/HTTP/Compression

Posting data in JSON format to the ASP.NET website using WinForm

Previously, we have shown how to post JSON data using JQuery to ASP.NET.

  http://laucsharp.blogspot.com/2013/03/posting-data-in-json-format-to-aspnet.html

Now, we are going to post JSON data using WinForm:

This is our business object which will reside at the server and client.

    public class Class1
    {
        public string code { get; set; }
        public string name { get; set; }

        public override string ToString()
        {
            return string.Format("code={0}, name={1}",
                this.code,
                this.name);
        }
    }

In the WinForm client program, when the user hit Button1 after keyed in the client code and name, the data will be submitted to the server:

        private void button1_Click(object sender, EventArgs e)
        {
            // store the user input into the business object.
            Class1 data = new Class1();
            data.code = this.client_code.Text;
            data.name = this.client_name.Text;

            // convert it into json format.
            JavaScriptSerializer js = new JavaScriptSerializer();
            string json_data = js.Serialize(data);

            // create the web request.
            HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://localhost:57655/dataGateway.ashx");
            request.ContentType = "application/json;";           
            request.Method = "POST";

            // write the json data into the request stream.
            using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
            {
                writer.Write(json_data);
            }

            // get the server response.
            using (WebResponse response = request.GetResponse())
            {
                // read the server response.
                Stream response_stream = response.GetResponseStream();
                using (StreamReader r = new StreamReader(response_stream))
                {
                    // do what ever you want with the response.
                    this.label5.Text = r.ReadToEnd();
                }
            }           
        }

Finally, at the server side, we add a Generic Handler (dataGateway.ashx) and it looks like this:

<%@ WebHandler Language="C#" Class="dataGateway" %>

using System;
using System.Web;
using System.IO;
using System.Web.Script.Serialization;

public class dataGateway : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        context.Response.ContentType = "text/plain";

        string s;
       
        // get the contents from the request stream
        Stream stream = context.Request.InputStream;
        using (StreamReader r = new StreamReader(stream))
        {
            s = r.ReadToEnd();
        }

        // ensure that the content is not empty.
        if (string.IsNullOrEmpty(s) || s.Length == 0)
        {
            context.Response.Write("'data' cannot be blank");
            return;
        }

        // convert it from json format to our business object
        JavaScriptSerializer js = new JavaScriptSerializer();
        Class1 obj = js.Deserialize<Class1>(s);

        // do whatever you want
        context.Cache["data"] = obj;

        // returns the response code/status to the caller.
        context.Response.Write("ok. received the data =>" + s);
    }

    public bool IsReusable { get { return false; } }
}

Next, sending compressed data in WinForm:

    http://laucsharp.blogspot.my/2018/04/posting-compressed-data-in-json-format.html

Errors that appear in the WCF client app...

I was setting a new server to host my WCF app and I hit the wall with the following error messages. I spent 2 days in solving this problem. I guess, many people is wondering how to resolve these interesting errors.

• The server has rejected the client credentials.
• The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:00:59.9687500'.
• The communication object, System.ServiceModel.ServiceHost, cannot be used for communication because it is in the Faulted state.

I have developed a WCF solution in WinForm which allows you to comment out certain settings in the server and client so that you can reproduce the above mentioned error message.

  https://github.com/lauhw/WcfWinForm

Please beware that if you are running the WCF server and client in the same computer, the above error might not appear until you run them in a separate computer.

In a nutshell:

1. Both server and client config file must have the same settings in the "Binding" record. This includes the "security mode".
2. The "identity" settings must be the same in both server and client config file.
3. In the live environment, make sure you remove the "MEX" endpoint unless you want anyone to be able to query the interface.
4. Don't forget to open the port in the firewall.

In case you are hitting the same error mentioned above, the fastest way to solve this problem is to run the test WCF solution (as mentioned above). Make sure that the test solution works. After that, compare the differences in app.config between the test solution and your solution.

Hope you don't have to spend 2 days to solve the configuration problem. ;)

Security check point

In Windows client, you can develop a static function call SecurityCheckPoint() that gets the user ID and password. This SecurityCheckPoint() method will be very useful whenever you want the user to re-authenticate before any process start OR you want to get the supervisor authentication.

Well, this can be useful for the web application as well but you need to implement it using JavaScript that show a modal dialog and AJAX calls for authentication.

Reusable class - user access control

This is another important area in the systems and it is reusable as long as the design is generic to cover all possible combination.

Basically, it should meet the following requirements:

• Security control - allows user to logon to the system and verify their user name and password.
• Access permission - permission could be controlled by menu level, screen level and field level (whether the field is show/hidden or enable/disable).
• User access log - this is compulsory for audit purpose and also the alert.
• Access deny alert - upon hitting certain number of invalid user name or password, the system should generate alert and email it to the system administrator.
• Password policy - you may consider to implement minimum password length and password complexity.
• Allow supervisor overriding - this can be useful when the current user does not have permission to access certain feature (such as edit customer address) but requires to do so.
• In case the system is a web base system, it should store the browser type and visitor IP address for audit purpose.

 You need the following classes to support the above mentioned requirements:

1. CUser - this class stores the user information such as user name, password, email address, etc.
2. CAction - this class stores all the permissions (or features) for the system.
3. CUserAccessLog - this class responsible for keeping track the user login and logout activities.
4. CLog - this class (which has been discussed in previous article) which is responsible for storing the audit information such as which "action" (or "featuer") the user has clicked.

For the detailed implementation, you might have work it out by yourself.

Reusable class - System log - the information for troubleshooting

In our system design, we are trapping all the exceptions that raised up at runtime and store it into a central database. This allows us to provide faster response time in fixing the error before the user complains. I know many people will ask why there is an error and why it was not catch at the development or testing phase. The answer is simple, we are not developing standard package. Almost all projects come with different requirements except for the "infrastructure" (such as the security, application log, etc).

To log down the exception, this error logging process should done in a very short time so that it won't affect the system performance or other processes. This can be achieve by saving the exception using a different thread.

In our system logging class, we have the following methods:

• AppendLog - this save the log as "audit log".
• AppendError - this save the exception and the failure point as "error".
• AppendWarning - this save the log as "warning" and it is very useful when the system setting is missing or misconfigure.

To track the failure point, refer to System.Diagnostics.StackTrace class.
To save the log information in another thread, refer to System.Threading.ThreadPool.

Our application log table design:

http://sqllauhw2000.blogspot.com/2012/07/you-need-application-log-for-your.html

Designing reusable control Part 2 - designing composite control

Of course, when things are simple, you can always try to look for the best available control and create a child class from it. But, in real life, you won't always get the easiest job. This is the time that you need more than inheritance from the object oriented programming.. the composite control.

Steps to develop a composite control using UserControl

1. Right click on the project.
2. Choose Add new item.
3. Select UserControl.
4. Then, drag the necessary controls from the common control section into the UserControl that you have just added.
5. Start writing some codes for this new UserControl. Of course, the most important will be loading data, validating user input and saving data. Another piece of code that is helpful is called "RefreshUI" which is responsible for controlling what should be hide or show on the screen base on the parent form/control. You might consider to add this method into your UserControl.

As simple as that, you have a composite control. Well, you can google for tutorial on the detailed steps with nice print screen. Sorry, I'm not going to repeat it. What I plan to do is to explain when do we need this?

From my experience, it seems many developers especially fresh graduate they "know" UserControl but "never use" it. Below is some example from live experience:

• My customer asked me to develop an attendance grid which displays all the check-in guests. So, I started with "attendance item" control which displays the guest's photo + name. Then, I developed another control call "attendance grid". Finally, I developed "attendance" control which relies on both attendance item and attendance grid.
• The situation gets complicated when you have domestic guests and international guests. These guests are using different "identity card" and the identity number are handled in different way. So, we develop a UserControl which is able to capture both domestic and international identity  number. This control is then reused in many of the screens.
• We also develop a composite control to show the necessary options for the user for generating report.
• Application menu is the composite control that will be reused in all projects.

Designing resuable control

Before we start the coding in any project, we have to identify the reusable screen, part of the screen or a group of fields. In this stage, we are developing custom control to be reuse in one or more projects.

Below is the reasons of developing custom control:

• To reduce the overall development time - in Visual Studio, the custom control can be drag from the Toolbox and drop onto any WinForm or UserControl. Note: the custom control that you have added to the project will appear in the Toolbox after you have compiled the project or solution (if your custom control is residing in other project).
• To reduce the testing time - this requires test once only because the control is reusable.

Below is the situation where you need to design a reusable control:

• You are working on system that requires to capture "record status" which is used in many screens. 
• The user might request for enhancement on this input control but the enhancement is not an urgent priority now
• You are splitting the system into multiple small section to be handle by different programmer.

For example, in your system, there is a field that occur in many screens and it is called "status". It contains 4 statuses.In this case, you may add a new Class into the project and then set the base class (or parent class) to "ComboBox". It has a method name RefreshData which is responsible for loading the available statuses (which could be from database and it's up-to your design).

public class StatusComboBox : ComboBox
{
    public void RefreshData()
    {
        this.Items.Clear();
        this.Items.Add("Pending");
        this.Items.Add("In Progress");
        this.Items.Add("Completed");
        this.Items.Add("Cancelled");
    }
} 

After you have compiled the project, the StatusComboBox custom control will appear in the Toolbox. Below is the screen snapshot of the Toolbox:

Now, you may drag this control onto the Form and call "statusComboBox1.RefreshData()" to load the items.

Splitting the big piece into small pieces

Whenever you get a project, don't be so excited and start writing codes. Below is some guidelines that helps you in managing the project development. Please take note that this article is not covering the entire SDLC (system development life cycle).

Development flow:

1. Ask for the sample reports and forms.
2. Designing the database - add the standard table structures such as security module, session table, etc. Then, follow by the new table structures.
3. Design the screen flow chart - you have to decide how many screens to the develop and the flow from one screen to another.
4. Identify the reusable class that can be shared with other project.
5. Identify the reusable custom control - most of the times, the screen (i.e., WinForm or WebForm can be splitted into multiple controls and then combine all the controls into one screen). This is very useful but requires thorough knowledge on the OOP (object oriented programming).
6. Design the screen layout.
7. Identify the shared functions - for example, the function to read the system settings out from the database or flat file.
8. Designing the report layout - the user might pass you the hard copy of reports. But, sometimes you might have to propose new format (in digital format) that includes the chart/graph to show the summarize data.
9. Start developing the system - now, you may start writing codes. I know you are waiting for this great moment.
10. Fine tuning the screen flow - some users might not understand why you have to design the screen in this or that way. So, you might have to fine tune the screen to reduce the support calls and training time. This is a very important step where most of the projects might failed here.
11. Fine tuning the field flow - this requires the feedback from the user where you might not have the full knowledge of the business.
12. Fine tuning the menu options - some of the menu option can be remove and replace by shortcut button or popup menu.
13. Add shortcuts or popup menu - I know this is not critical to any system but you should always do this. Most of the users will appreciate your effort in this area and this is the surprises when they found out there is an easier and convenient way to do their job. For example, the user may press the Delete key or right click and choose Delete in order to delete the selected files in Windows Explorer.
14. Fine tuning the query response time by add appropriate indexes or using multi-thread - this is especially loading lots of records from the database or generating reports. Do you know how frustrated is that for the user to see "not responding" word appear in the program/task manager?

Assumptions:

1. Always assume that you are not sure about the detail process - so ask for all the details and do not assume what you know is correct.
2. Always assume that the user still have something exceptional procedures to follow - well, you have to prepare yourself before interviewing the users.
3. Always assume that the user will request for changes in the database or system flow - this requires declaring constants or reading the settings from the database, writing modular program, passing the parameter with an object type instead of value type, etc.
4. Always assume that the data will grow faster than the user is expecting - when the system has rolled out, it will grow faster than you are expected. The proper way to handle this is to calculate the number of transactions per week/month that will be entered into the database. Then, estimated the number bytes per transactions. Base on the total bytes that requires, you will be able to come out with an appropriate server/workstation specification.

 Is that all? NO. You have to learn from experience or from other experts and then build your own checklist.

Embedding WebBrowser control in WinForm

This is not something new but very useful in embedding the WebBrowser control in WinForm:

http://notions.okuda.ca/2009/06/11/calling-javascript-in-a-webbrowser-control-from-c/

Pros:

1. Multi-media mash up without much GDI calls - you can add nice images, sound and video in your WinForm without having to use GDI api to draw the nice effect.
2. Able to utilize JQuery (or JavaScript) - this allows you to develop application that shares some of the library that you have developed between the WinForm and ASP.net.
3. Able to do fancy menu without going through the WPF/Silverlight route - just use all the HTML and JavaScript that you have already learned.
4. No more "can't built nice control" any more.

Cons:

1. Over-used of the JavaScript and HTML in WinForm might create chaos. It increases the difficulties in debugging.
2. You must posses HTML, JavaScript and C# skills. Of course, you must have the SQL skill as well.

Detecting the parent Form from Component class

Component is a class that allows you to build non-visual control. For example, Timer class. By the problem is that it does not have a intuitive way to get the parent control.

Below is the code that will get the parent control reference and save the reference during the design time. Please take note the following code will work when you are dragging a control from the Toolbox onto the Form. For those Component that has already dropped on the Form before you added codes below, it will not work.

private ContainerControl _containerControl = null;
        public ContainerControl ContainerControl
        {
            get { return _containerControl; }
            set { _containerControl = value; }
        }

        public override ISite Site
        {
            get { return base.Site; }
            set
            {
                base.Site = value;
                if (value == null)
                {
                    return;
                }
                
                IDesignerHost host = value.GetService(typeof(IDesignerHost)) as IDesignerHost;
                if (host != null)
                {
                    IComponent componentHost = host.RootComponent;
                    if (componentHost is ContainerControl)
                    {
                        ContainerControl = componentHost as ContainerControl;
                    }
                }
            }
        }

AsyncOperationManager for WinForm

To run a process in a thread that is other than UI, you may use the following:

1. BackgroundWorker class - allows you to post the progress in integer type only.
2. AsyncOperationManager class - allows you to post any object into the method that is running in the UI context.

If you need to pass the information to be shown in the UI, AsyncOperationManager is much more flexible.

Below is the sample code on using the AsyncOperationManager class.

   private void button3_Click(object sender, EventArgs e)
   {
       AsyncOperation ao 
             = AsyncOperationManager.CreateOperation(null);

       Thread t = new Thread(new 
                        ParameterizedThreadStart(DoRun2));
       t.Start(ao);
   }

   public void DoRun2(object state)
   {
       AsyncOperation ao = (AsyncOperation)state;

       int j;

       for (int i = 0; i < 100; i++)
       {
           j = tl.id;
           ao.Post(this.DoProgress2, i);
           System.Threading.Thread.Sleep(90);
       }
 
       ao.OperationCompleted();
   }
   void DoProgress2(object state)
   {
        // you don't have to call label2.Invoke() method because 
        // ao.Post() will put the context back to the UI 
        // context automatically.
       this.label2.Text = state.ToString();
   }

How to upload file using WebClient

To use System.Net.WebClient class, first, you have to create an ASPX page in the website as shown in MSDN website. Then, execute the following:

System.Net.WebClient cli = new System.Net.WebClient();

cli.UploadFile(url,
"POST",
file_name);

Reference:
http://msdn.microsoft.com/en-us/library/system.net.webclient.uploadfile(VS.71).aspx
http://www.developerfusion.com/forum/thread/38843/
http://www.csharpfriends.com/Articles/getArticle.aspx?articleID=115