Monday, July 27, 2015

Microsoft Azure Webjobs

Microsoft Azure Webjobs

The long awaited webjobs is now in preview for Windows Azure!  How many times have you deployed a website, then after a couple months requirements change and they want you to run a daily or continuous job that processes files or re-size images or works a queue, now with web jobs in Windows Azure it is easy. 

What is webjob?

Azure WebJobs enables you to run programs or scripts in your website as background processes. It runs and scales as part of Azure Web Sites. It is kind of a lightweight Worker Role associated with web site. Process life cycle is strictly connected with web sites, so restarting a web site will cause a Web Job restart.

Each Web job can be configured to run in one of the following modes:
1. Continuous – runs all the time, analogy to Windows Service configured with auto start option is in place. Hosting environment monitors the job status and brings it up when process is down. (NOTE: With the current preview, this works properly on standard web sites with ‘always on’ setting. Caution is required when we scale the site up from Standard mode – Web Jobs will be terminated when there is no request for 20 minutes)
2. Scheduled – runs at particular times.
3. On Demand – runs when started manually.

Azure WebJobs can be created using the following scripts: 
  1. .cmd, .bat, .exe (using windows cmd)
  2. .ps1 (using powershell)
  3. .sh (using bash)
  4. .php (using php)
  5. .py (using python)
  6. .js (using node)

Creating WebJobs:
A simple console application will work for you as a webjob.

Deploying Webjobs:
We can deploy webjob in either two ways -
1. Automatic deployment - When you publish web application, webjobs get deployed automatically.
2. Manual deployment - Manually deploy code file to portal.

Manual Deployment:

1. To manually deploy our application as new Web Job – we need to create a zip file that contains our project output (in our case standalone exe file is enough). It is important to remember that the main executable file needs to be on the root level of that zip file. Name of the file is irrelevant. The current limit size for one zip file is 100 MB.

2. Open Azure Management Portal and select destination web site.
3. On the web site details page go to Web Jobs tab and click New
4. Now we can configure our new Web Job.I want to run this one every 5 minutes:


5. Now, our job is up and running. To verify it, we can check log file produced by Web Job (link in Logs columns) – this file contains diagnostic information about run and application output that was redirected automatically to that file (the same goes for error stream, so if, by any chance, program throws exception that is unhandled it will also be logged in that file and job run status will be set as Failed)


And the output:


Automatic Deployment:
1. To deploy the WebJob, right-click on the project and select Publish as Azure WebJob.

2. Select when the WebJob shall run and click OK. In this example, the job shall be run on demand.


3. Select a publish target, in this example the target shall be an Azure WebSite

You shall be required to sign-in and enter your Microsoft Azure credentials

Once signed in, you may select in which WebSite the Job Shall runs.

4. Click on OK, then select Publish.
Now, if you go the the Microsoft Azure portal, and navigate to your WebSite, you shall see a new WebJob created and deployed.
Make webjob settings configurable:
It attempts to read configuration settings in this prioritized order:
  1. Try to find the configuration value in the Web Site's online configuration (see below).
  2. Try to find the configuration value in the .cscfg file.
  3. Try to find the configuration value in the app.config file or web.config file.
By the Web Site's online configuration, I mean the configuration settings you can edit on a deployment-by-deployment basis using the Azure management portal, under the Configure 'tab':

Online configuration for an Azure Web Site
Exception in webjobs which I have faced:
Azure WebJob that fails after only 20 minutes of running with the exception "Thread was being aborted".
Cause: By default, web apps are unloaded if they are idle for some period of time. This lets the system conserve resources.
Resolution: In Basic or Standard mode, you can enable Always On to keep the app loaded all the time. If your app runs continuous web jobs, you should enable Always On, or the web jobs may not run reliably.
Also add few settings to your webjob which are mentioned below -
                                   The time is specified in seconds

Web Job: Azure web job aborted some time

Issue: If deployed azure webjob running for long time, it will get aborted after some time.

Issue cause: Sometimes there are interruptions which can stop your WebJob abruptly without notice aborting your process, and get time out.
These interruptions could be due to: stopping your site, restarting your site, some configuration change to your site which causes your site to

restart, Azure maintenance (version update for example) or even the VM simply crashing for some reason.

Make confuguration on for - ALWAYS ON = ON.

Go to Configure menu under APP settings add follwing:
- SCM_COMMAND_IDLE_TIMEOUT = 90000 (in seconds)
- WEBJOBS_IDLE_TIMEOUT = 90000 (in seconds)

- stopping_wait_time = 90000 (in seconds)

Thursday, June 18, 2015

ServiceConfiguration transformation

Add following sections in cloud solution ccproj
<ServiceConfiguration Include="ServiceConfiguration.production.cscfg" />
<ServiceDefinition Include="ServiceDefinition.csdef" />
<ServiceConfiguration Include="ServiceConfiguration.Dev.cscfg" />
<ServiceConfiguration Include="ServiceConfiguration.Int.cscfg" />
<ServiceConfiguration Include="ServiceConfiguration.Staging.cscfg" />
<ServiceConfiguration Include="ServiceConfiguration.Local.cscfg" />
<ServiceConfiguration Include="ServiceConfiguration.Cloud.cscfg" />


<PropertyGroup Condition=" '$(Configuration)' == 'Dev' ">
<PropertyGroup Condition=" '$(Configuration)' == 'Int' ">
<PropertyGroup Condition=" '$(Configuration)' == 'Staging' ">
<PropertyGroup Condition=" '$(Configuration)' == 'Production' ">

<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v12.0\Web\Microsoft.Web.Publishing.targets" />
<EnvironmentDefinition Include="ServiceDefinition.Debug.csdef">
<EnvironmentDefinition Include="ServiceDefinition.Production.csdef">
<EnvironmentDefinition Include="ServiceDefinition.Int.csdef">
<EnvironmentDefinition Include="ServiceDefinition.Local.csdef">
<EnvironmentDefinition Include="ServiceDefinition.Dev.csdef">
<EnvironmentDefinition Include="ServiceDefinition.Cloud.csdef">
<EnvironmentDefinition Include="ServiceDefinition.Staging.csdef">
<None Include="@(EnvironmentDefinition)" />
<Target Name="ValidateServiceFiles" Inputs="@(EnvironmentDefinition);@(EnvironmentDefinition->'%(BaseConfiguration)')" Outputs="@(EnvironmentDefinition->'%(Identity).transformed.csdef')">
<Message Text="ValidateServiceFiles: Transforming %(EnvironmentDefinition.BaseConfiguration) to %(EnvironmentDefinition.Identity).tmp via %(EnvironmentDefinition.Identity)" Importance="High" />
<TransformXml Source="%(EnvironmentDefinition.BaseConfiguration)" Transform="%(EnvironmentDefinition.Identity)" Destination="%(EnvironmentDefinition.Identity).tmp" />
<Message Text="ValidateServiceFiles: Transformation complete; starting validation" Importance="High" />
<ValidateServiceFiles ServiceDefinitionFile="%(EnvironmentDefinition.Identity).tmp" ServiceConfigurationFile="ServiceConfiguration.$(Configuration).cscfg" />
<Message Text="ValidateServiceFiles: Validation complete; renaming temporary file" Importance="High" />
<Move SourceFiles="%(EnvironmentDefinition.Identity).tmp" DestinationFiles="%(EnvironmentDefinition.Identity).transformed.csdef" />
<Target Name="MoveTransformedEnvironmentConfigurationXml" AfterTargets="AfterPackageComputeService">
<Copy SourceFiles="ServiceDefinition.$(Configuration).csdef.transformed.csdef" DestinationFiles="$(OutDir)ServiceDefinition.csdef" />

<Target Name="PublishVirtualApplicationsBeforeCSPack" BeforeTargets="CorePublish;CsPackForDevFabric" Condition="'$(PackageForComputeEmulator)' == 'true' Or '$(IsExecutingPublishTarget)' == 'true' ">
<Message Text="Start - PublishVirtualApplicationsBeforeCSPack" />
<PropertyGroup Condition=" '$(PublishDestinationPath)'=='' and '$(BuildingInsideVisualStudio)'=='true' ">
<!-- When Visual Studio build -->
<PropertyGroup Condition=" '$(PublishDestinationPath)'=='' ">
<!-- When TFS build -->
<Message Text="Publishing '%(VirtualApp.Identity)' to '$(PublishDestinationPath)%(VirtualApp.PhysicalDirectory)'" />
<MSBuild Projects="%(VirtualApp.Identity)" ContinueOnError="false" Targets="PublishToFileSystem" Properties="Configuration=$(Configuration);PublishDestination=$(PublishDestinationPath)%(VirtualApp.PhysicalDirectory);AutoParameterizationWebConfigConnectionStrings=False" />
<!-- Delete files excluded from packaging; take care not to delete xml files unless there is a matching dll -->
<CreateItem Include="$(PublishDestinationPath)%(VirtualApp.PhysicalDirectory)\**\*.dll">
<Output ItemName="DllFiles" TaskParameter="Include" />
<FilesToDelete Include="@(DllFiles -> '%(RootDir)%(Directory)%(Filename).pdb')" />
<FilesToDelete Include="@(DllFiles -> '%(RootDir)%(Directory)%(Filename).xml')" />
<Message Text="Files excluded from packaging '@(FilesToDelete)'" />
<Delete Files="@(FilesToDelete)" />
<Message Text="End - PublishVirtualApplicationsBeforeCSPack" />

Tuesday, April 28, 2015

Azure : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.

Problem: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.

In azure, while running the project in temporary folder it creates your azure project and its related files. some dll names are large so in result whole path is large and which results in above error.

There are 2 solutions to solve this issue:

Solution 1:
Step 1: Open .ccproj project
Step 2: Under PropertyGroup Add following line

Source :

Solution 2:
Step 1: Right click on "My Computer" -> Select Properties
Step 2: Click on "Remote Setting" -> Select Advanced -> Select Environment variables -> Click on "New" -> Add Variable name as "_CSRUN_STATE_DIRECTORY"
and Add Variable value as whatever you want. ex: C:\A -> click on save

Friday, April 24, 2015

The underlying connection was closed: An unexpected error occurred on a receive.System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host

The underlying connection was closed: An unexpected error occurred on a receive.System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)

To resolve above error add following properties to webrequest object
HttpWebRequest webRequest = (HttpWebRequest) base.GetWebRequest(uri);
            webRequest.KeepAlive = false;            webRequest.ProtocolVersion=HttpVersion.Version10;

Friday, April 10, 2015

JSON to Dataset conversion

Code snippet :

public DataSet ConvertJsonToDataSet(string jsonString)

                XmlDocument xd = new XmlDocument();
                jsonString = "{ \"myrootNode\": {" + jsonString.Trim().TrimStart('{').TrimEnd('}') + "} }";
                xd = (XmlDocument)JsonConvert.DeserializeXmlNode(jsonString);
                DataSet ds = new DataSet();
                ds.ReadXml(new XmlNodeReader(xd));
                return ds;

Tuesday, March 31, 2015

Filename convenventions while uploading files.


When you attempt to create, save, or rename a file, folder, or shortcut, you may receive one of the following error messages:
A filename cannot contain any of the following characters:\ / : * ? " < > | or This filename is not valid
Resolution :
To create, save, or rename a file, folder, or shortcut, use a valid character, dont use following charachters:
 \ / : * ? " < > |
Characters that are valid for naming files, folders, or shortcuts include any combination of letters (A-Z) and numbers (0-9),
plus the following special characters:
   ^   Accent circumflex (caret)
   &   Ampersand
   '   Apostrophe (single quotation mark)
   @   At sign
   {   Brace left
   }   Brace right
   [   Bracket opening
   ]   Bracket closing
   ,   Comma
   $   Dollar sign
   =   Equal sign
   !   Exclamation point
   -   Hyphen
   #   Number sign
   (   Parenthesis opening
   )   Parenthesis closing
   %   Percent
   .   Period
   +   Plus
   ~   Tilde
   _   Underscore