Dll Shepherd.Net

Leaving a Better Place

2013-06-11T13:19:00.001-07:00

I never actually wrote it in this blog but for the last 3 years I have been a Better Place employee (I have actually had an image of an award we received back in 2011 with the words Better Place but never actually wrote them as text).

It’s actually been a great 3 years where I met some amazing people and learned great technologies. When I joined the team our system – GTM (Grid Topology Management) was just starting up – basically the only action available was creating Sites (in Better Place the Site entity was the main entity – it could either mean a charging Site or a Battery Switch Site (BSS)). The server side at that time had a single WCF file with all the code (logical and DB access, and with using ArcObjects meant quite a bit of code).

It didn’t take me (too) long to change all that so the server side (which is more my specialty) now has a separation between the service layer, the BL (Business Logic) and the DAL (Data Access Layer). The BL is fully tested using NUnit (for a time it was VSTests). The Silverlight UI had so many upgrades like:

Google Street View from the Silverlight Application
Editing the Electric Topology (adding a Panels, Circuits, Spots ,Sockets and sometimes even communication elements since Better Place’s Grid could be remotely controlled – and in Denmark end users could set when their Electric usage was at a peak so that the Electric Vehicle will get less).
Electric Topology validation – so that before installation all the planned hardware elements could be verified to be correct
Routing – going from point A to B when you have to look out for your battery or switch it is not something Waze implemented (though wouldn’t it be cool if you just entered your remaining KMs until you have to fill your tank or switch your battery and have Waze do it automatically – with your preferred Gas Station (or BSS))
Traffic Reports – basically like it is implemented in Google Maps (only we used a paid service

The server side had many integration services build as well like making an order from SAP, communicating to the Electric Vehicle what Sites are down (maintenance or some kind of technical problem) and much more.

Along the way I somehow got some free software like Resharper, dotTrace (maybe I should be a QA?) and NDepend.

My last year at Better Place was dedicated to lowering operation costs and improving the system. One of those improvements was moving from SQL Server to Oracle, which considering we were using Linq2Sql meant moving to another Framework. For a time our team used Entity Framework and another team at the department used NHibernate, than for solidarity reasons we moved to NHibernate (I might write a blog post on that because: A. we had to use Timestamp and NHibernate “supports” it by using DateTime – not DateTimeOffset. B. for some odd reason NHibernate kept adding semi random numbers to column names’ aliases and with Oracle’s query optimizations that caused problems unless we used projections). The mapping for our NHibernate usage was done using Fluent NHibernate (for some odd reason I never liked writing xml where I could use plain code, is it just me?).

Through the downsizing and people leaving us I was put in charge of the in car computer called Oscar, unfortunately I only had enough time too learn it’s systems – not make any changes. I did find many bugs in the Media Player system (I even wrote an unofficial guide about how to operate it with the least amount of problems, like did you know playing all the songs on your DOK will be changed the next time you start your car to playing only the last artist’s songs?).

In the last 3 months we also had a MVC+jQuery course and had started to implement the new Self Service site (you can see it’s design here) when as the papers say “the battery has run out”…

Today I had to return my Electric Vehicle, yesterday I switched the battery for the last time with this vehicle (if the service continues I might just get another one sometime in the future, I really never had such good performance from any other car).

So after this long but partial summary of the last 3 years it is time to say I have started looking for the next Better Place. If you are looking for a Software Engineer (even if it’s not map related) then you are welcome to email me with the details. My email is written on the right.

ORA-03149: Invalid Oracle error code

2013-04-08T05:07:00.001-07:00

Just got this error.

The only thing I saw on the web on this was: “This is an internal protocol error. Contact Oracle Support Services.”

As I see it, it means Oracle doesn’t know what I did wrong.

At the end we found out that the error was caused by entering a value of 1.7976931348623157E308 to a NUMBER(38, 8) field. The person that sent me this value meant to send NaN but at the end sent Double.MaxValue.

I am sure this is just one symptom of the problem but maybe it will help someone out there…

SDE: Using SQL to copy feature rows

2013-03-27T04:58:00.001-07:00

(Or to Bulk Insert features to a feature table)

This only works if you are using Oracle/MS SQL geometry types (where the shape field is not an integer but an actual shape).

I always thought that for copying more than one row of feature data I will need to use ArcCatalog, for one row I usually hard coded the OBJECTID field and hoped that the next person using ArcCatalog won’t hate my guts…

Well last week I needed to duplicate rows in a feature table and found this link, and even though it says ArcSDE 9.3.1 as the latest version it works on version 10.

Use:

SELECT table_name, registration_id FROM sde.table_registry WHERE owner = 'OWNER'

To find the registration number of the feature table, for example 21.

To find the next available ObjectId:

sde.version_user_ddl.next_row_id('OWNER', 21)

To copy the rows of Table_Name (for example: from type 1 to type 2):

insert into Table_Name(objectid, shape, Type) select sde.version_user_ddl.next_row_id('OWNER', 21), shape, 2 from Table_Name t where Type = 1

And that's it.

HTML5+MVC Course, Day 3

2013-03-04T02:32:00.001-08:00

Continued from day 1, day 2.

This lesson was more a review of the last lesson with the added value of looking at profiling tools. In the summary I just skipped the review part…

After opening a new MVC project from a template looking at the libraries installed you will find that they are old libraries. You can update them if you wish using NuGet –> Update:

NuGet knows which libraries it needs to update by looking at the packages.config file that describes the installed packages and their versions. For example:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="EntityFramework" version="5.0.0" targetFramework="net45" />
  <package id="jQuery" version="1.7.1.1" targetFramework="net45" />
  <package id="jQuery.UI.Combined" version="1.8.20.1" targetFramework="net45" />

Web Programming must have:

Browser Sniffer with a capability to see the HTML – Chrome F12
Client side Sniffer that displays the data as it passes (optional: with JSON parser, with debugging, comparison) – Fiddler
Server side: DB profiler for queries to the DB – SQL Server Profiler

Fiddler

Clicking on the “Capturing” in the bottom left corner (pink arrow) will stop and start the capturing.

Clicking on the X (orange arrow) will give you in it’s first option: Remove all. Clears the screen.

HTTP Results:

HTTP 100-199 are for information
HTTP 200-299 are for successful result, for example: 200 is for success (green arrow)
HTTP 300-399 are for redirect, for example: 304 (blue arrow) indicates the result is in browser’s cache
HTTP 400-499 are for client errors, for example: 404 page not found
HTTP 500-599 are for server errors, for example: 503 server unavailable

On the top-right side of the screen Inspectors->raw we can see the request with it’s http headers

On the bottom-right side of the screen: raw if the message is encoded you will see gibberish (and the Content-Encoding will be gzip):

HTTP/1.1 200 OK
…
Content-Encoding: gzip
…
Content-Length: 44892

��`I�%&/m�{J�J��t��`$ؐ@��iG#)�*��eVe]f@�흼��{��{��;�N'��?\fdl��J�ɞ!��?~|?"��O�<y��<M��<��ⳏ��

*** FIDDLER: RawDisplay truncated at 128 characters. Right-click to disable truncation. ***

Then:

Click on “Response is encoded and may need to be decoded before inspection. Click here to transform” (blue arrow). Or do it by hand by going to the Transformer tab and select “No Compression” in HTTP Compression.

Ajax+JSON Calls:

You can identify Ajax calls by:

Ajax calls will have: X-Requested-With: XMLHttpRequest (blue arrow).

JSON calls will have: Accept: application/json (green arrow), we can also see that the response has the Content-Type we requested application/json (bottom row).

The JSON response can be read several ways:

1. Text/Raw view:

[{"value":"ALFKI","label":"Alfreds Futterkiste"},{"value":"ANATR","label":"Ana Trujillo Emparedados y helados"},{"value":"ANTON","label":"Antonio Moreno Taquería"},{"value":"AROUT","label":"Around the Horn"}]

Raw will also show the headers.

2. JSON view:

Views the JSON as a data structure – more readable.

Chrome F12 Profiler

Console

From the Console you can run jQuery queries like you can run code in the Immediate of VS:

Console will also show JavaScript errors:

We will know there is an error by the error icon at the bottom-right corner (red arrow).

We will get the exception details like: from where it was thrown and the description at the center (orange arrow).

And most importantly we will get a link to the actual location in the code the error was thrown from (blue arrow), clicking on it will lead us to _ window:

As you can see the error icon appears on all screens (red arrow).

You can see the location of the error (yellow arrow).

And you can watch the local variables at the right side (blue arrow).

You can also add more functionality with extensions (for example PageSpeed).

Timeline

very important!

At the bottom there is a button for starting the profiler (red arrow).

At the top you can select the timeframe, my browser default for some unknown reason was the first 1 second – so the default is not seeing anything (use the lines at the blue arrows to adjust the time frame).

Profiles: Memory leak checker

Take heap snapshot at the start and you get the current memory profile:

As with most memory profiling tools you can sort (blue arrow) by object count, shallow size (the size of the element by itself), retained size (the size of the element including links to other classes). You can read more on the size differences here, but we will use retained size.

You can also drill down to the actual function/line that takes the memory (green arrow).

Take another snapshot and you can compare snapshots:

At the bottom change from Summery to Comparison (blue arrow) and all the view changes to a delta between the runs.

Note: IE has it’s own profiler but it seems to be for running time and not for memory leaks.

Entity vs. Class

Entity has to have a unique identifier – Key.

Data/Class is without an identifier, JSON uses data not entities (only Silverlight has entities in the client)

If we were to return an Entity with JSON:

public JsonResult GetCustomersCircularError(string term)
{
    var results = from c in db.Customers
                  where c.CompanyName.StartsWith(term)
                  select c;

    return Json(results.ToList(), JsonRequestBehavior.AllowGet);
}

The parser might fail with a circular reference exception:

Instead we have 2 options:

1. POCO: Create a class (View Model) for the return value – one without circular references.

public class JsonCustomer
{
    public string value { get; set; }
    public string label { get; set; }
}

public JsonResult GetCustomersClass(string term)
{
    var results = from c in db.Customers
                  where c.CompanyName.StartsWith(term)
                  select new JsonCustomer { value = c.CustomerID, label = c.CompanyName };

    return Json(results.ToList(), JsonRequestBehavior.AllowGet);
}

2. Use Linq+Anonymous Classes to return that data from the DB entities:

public JsonResult GetCustomers(string term)
{
    var results = from c in db.Customers
                  where c.CompanyName.StartsWith(term)
                  select new { value = c.CustomerID, label = c.CompanyName };

    return Json(results.ToList(), JsonRequestBehavior.AllowGet);
}

Yair believes the second option is best because the class will only be used for returning values (you can’t use it in the client because the client is in JavaScript – not C#).

Demo: AutoComplete+Redirect

AutoComplete box that when an item is selected the page is redirected to another and the value of the AutoComplete is passed to the new page.

Server side:

public ActionResult OrdersForCustomers(string id)
{
var orders = db.Orders.Where(ord => ord.CustomerID == id).Include(o => o.Customer);
return View("Index", orders.ToList());
}

Client side:

We will use the select event of AutoComplete box. Since we don’t really know the event arguments lets debug them to see their values:

<body>
    <input id="txtCustomer" />
    <script>
        $("#txtCustomer").autocomplete({
            source: "orders/GetCustomers",
            select: function (event, ui) {
                alert("Breakpoint here!");
            }
        });
    </script>
</body>

Note: putting the script tag inside the HTML body is another way to write $(document).ready(function(){…

We will add a breakpoint at the alert to see the argument values:

As we can see the event argument contains information on the event itself: who called it, from where on the screen, the control name…

The ui argument is what we need it contains both the value and the label of the item selected.

Now the only thing left is to do the redirect:

And the page will change to the correct URL.

HTML5+MVC Course, Day 2

2013-02-25T05:57:00.001-08:00

Continued from day 1.

Last class we learned about the client side, today we are going to go over the server and talking to the server. In the past it was done with Forms and the method Post. Today there is Ajax.

If you want to skip the history lesson you can just go straight to “ASP.NET MVC 4” title.

General

There are 3 option to build Web Applications:

1. Html rendering in the Server side

Every action from changing the color of the text to filtering a table is done by calling the server and getting the result to the client. If you wanted to see the Current Movies, Current Traffic, something from the DB/File system the server renders it.

The server is a “Factory” for Html.

Technological History: CGI, ASP (allowed saving data in the server using the Session object), Sun did something similar with Java.

The Code is build into a Dll, you get to write in a real language (.Net or Java), you get memory management, variables, …

You want to do something, you fill out a Form and data goes to the server. But doing something like Filtering required going to the server as well. Microsoft added a lot of libraries to help this pragma: Rendering, Caching, Database, Controls. All of them for the server side.

The Browsers became stronger, so they moved to option 2:

2. Ajax: Html Rendering in the Server but some parts call the server for updates

In the past there was a Frame that you could use to receive updates to just part of the page (most of the Portals inside organizations are still built like this). But Frames are in the past and HTML5 doesn’t include them. Frames are windows that work independently – like opening a different browser per frame, but with frames you can have one frame ask another to do some action. Every frame has a different DOM.

Inner objects in the Html page are calling the server with Ajax to receive updates. The whole page is not updated just the inner part.

Ajax Call – Microsoft has invented a component called XML Http (later the other Browsers implemented it as well) that is an Object that lives in the DOM (it is actually part of the DOM), it resembles a Frame, it has an inner Form, it sends a request and gets back a response asynchronously (without asking – it’s default way). XML Http was later renamed Ajax, they first believed that you will just transfer XML data and in the client transform it using XSL – joining them both will result in HTML (it was called B2C - Business to Consumer). For a long time no one used that object because ASP got in and made use of option 1, only a few people actually worked with it.

Then Mozilla-Firefox and Google-Chrome adopted this object and gave it the name Ajax - Asynchronously JavaScript And Xml: Asynchronously because that’s how it worked, JavaScript because that the language that consumed it, And Xml that was back then. Today the only part left is asynchronously (JavaScript is a given) no one uses Xml…

ASP.NET Ajax is Microsoft adoption of option 2: it uses a hidden field named ViewState to pass data from the server to the client, they built the ViewState in the Client and passed it to the Server, then built in the Server and passed it to the Client… It worked well: Render in the server and update increments. Sorting some table for example sent Ajax to the server, the server returned a small Html part, then you took a “knife” and cut out the old Html and put instead the new Html. All you had to do was mark which Control you wanted and the system would disable the submit for those controls, do an Ajax request, receive an Ajax response with just the Html needed and replace the control.The problem is it was all done automatically, architecturally wrong: just “drag and drop” and the system patched all the component together. This system was easy for starting users but annoying to advanced users: even changing the design of a button might cause headaches.

ASP.NET came in 2001 to solve the then problem of memory. The Session was a problematic object (the Enemy) that was kept for 20 minutes (by default). So to solve that problem ASP.NET transformed the session to an object that was passed from the server to the client called the ViewState. Since that time memory became less of an enemy, storage has became cheaper but Microsoft was stuck with the ViewState (not using it was an equitant to going back to ASP).

Then came PHP that Facebook was built using a spinoff of PHP. PHP came with the attitude of not using the ViewState hidden field. After that in 2010 Microsoft came up with ASP.NET MVC.

2.1. ASP.NET MVC

The Controller (written in C# and can be debugged and unit tested) creates/puts data into the View and renders the Html in the Server. The controller also allows returning JSON to the client without rendering the Html (the + 0.1). As opposed to ASP.NET Web Forms:

No more Ajax
Works with jQuery (in it’s orientation)
Total separation between the client side and the server side

No longer uses the ViewState, instead uses the Session.

In 2012 Microsoft adds ASP.NET Web API, Mobile, Facebook extension,…

3. Html Rendered in the Client, the Server supplies the data

The application is written with a static Html page, the Server supplies the data with the Business Model. No Html is rendered in the Server. The rendering is done in the Client. The first template might come statically from the server but nothing is rendered in the server.

jQuery was build in the same time as ASP.NET Ajax by John Resig, started as a library that supplied selectors by Id, Class name, Element name, etc. (Original JavaScript code). Minimal code that rely on CSS3. Supports all the Browsers (from IE8 and on, Firefox, Chrome…). Then he added some elementary actions like toggle, style, etc. And (of course) provided support for Ajax (in a property that has a url to which you can GET or POST with a name-value array).

jQuery became a Must Have for every Web Developer in the industry. On top of that came the jQueryUI group that build it’s control on top of jQuery as an alternative to Microsoft’s ASP.NET Ajax (like the AutoComplete box that is available at both libraries).

At around that time JavaScript Framework started appearing that enabled OOP JavaScript, with MVVM or MV* (MV Something): Model you always need, View you always need, now you need something that connects them – in MVC it’s the controller and in MVVM it’s the View Model. The Frameworks were built in GitHub where entire communities were built around those projects. One of those is Knockout by Steve Sanderson who is now a Microsoft employee (though the code is not owned by Microsoft). Knockout gives Templates and is very similar to Silverlight.

ASP.NET MVC 4

We will start with ASP.NET MVC of option 2 (fully rendering in the server side: looking briefly at the Controllers of MVC, the routing mechanism, the view, the Razor engine that help the view. The model will be built using Entity Framework. First by hand then using a wizard) and upgrade it to option 3 (with a html page that calls the server for JSON data, with jQueryUI AutoComplete).

Create empty MVC Project with Empty Project Template (or just use the one from last lesson):

Add Controller named HomeController – by right clicking the Controllers folder-> Add-> Controller… (or Ctrl+M,Ctrl+C):

Template: Empty MVC controller

Note: Controller names must end with Controller, MVC at the server works with naming conventions.

In HomeController we get a default method of Index():

Trying to run it will return an error “The view 'Index' or its master was not found or no view engine supports the searched locations. The following locations were searched…”:

The method:

public ActionResult Index()
{
return View();
}

leads us to the default View of the method named Index. MVC looks for the view Home-Index at a lot of location but doesn’t find it, because it doesn’t exist we get an Error. We have a Controller Home with Index that returns View() – which is the default view for that action.

Another way of thinking about it is: The Controller is C# code, the View is HTML script. The code we have cannot be translated to HTML.

The View method has several overrides:

View() – default view: the same name as the method that calls View(), in our case Index
View(object) – object that he can shove into the default view as it’s data (in the future it will be the Model)
View(IView) – using a custom view object
View(IView, object) - using a custom view object with object data
View(string) – the value of the string is the name of a different view that will be the View
View(string, object) – view name with object data
View(string, string) – view name with master name
View(string, string, object) - view name with master name and object data

Adding a View is as easy as right clicking the Index method and clicking Add view (or Ctrl+M, Ctrl+V):

For now we will uncheck the “Use a layout or master page”, because we don’t have one. The view was added at:

View\Home\Index.cshtml

(the location is a very important part)

Now running the application won’t cause any error. But the View is static, the Controller doesn’t do anything.

Note: CSHTML (the extension of the view file) doesn’t have a built in designer. You can use a designer to build the HTML and just copy paste that to the CSHTML.

Passing data from the Controller to the View:

For that we will use a dynamic object named ViewBag that is defined in MVC.

In the Controller writing:

ViewBag.Msg = "Hello";

In the View:

<body>
    <div>
        @ViewBag.Msg
    </div>
</body>

The result is a page displaying Hello.

Configuring the default page in MVC:

AppStart\RouteConfig.cs

Contains the code that tell MVC that the default Controller is Home and the default Action is Index:

public class RouteConfig
{
    public static void RegisterRoutes(RouteCollection routes)
    {
        routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

        routes.MapRoute(
            name: "Default",
            url: "{controller}/{action}/{id}",
            defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
        );
    }
}

The first row just tells the server to ignore all requests with .axd. Because ASP.NET uses urls with .axd extension internally.

The second row tells us what the default url is – Home\Index. it also tells us that it receives an Id and that it is optional.

Note: the Home Controller is the default Controller (it isn’t because we added a Controller named Home, it was like this before hand). And Index is the default action.

Passing Id to an Action:

In the Controller:

public ActionResult Test(int? id)
{
ViewBag.Msg = id.HasValue ? id.Value.ToString() : "Hello test";
return View("Index");
}

View(“Index”) tells MVC to use the Index view for this action.

Calling from the Browser:

\Home\Test\123

(Better for GET request if you need the extra length of GET)

\Home\Test?id=123

Note: Passing something that isn’t an int (like ABC) to Test will be the same as not passing a value, i.e. passing null.

What is better Get or Post?

GET wakes the server and does the request at the same time.

Limited in it’s length.
Security: The talk about being insecure is wrong since POST without encryption is just as insecure as GET (passing a user+password in the url is the same as passing it in POST since using any sniffer you can see the variables, it should be preferred to put it in the url so that it could be fixed).
If you can get – GET: Much better at the performance. Google uses GET.

POST calls the server, the server opens a socket, the server returns HTTP 100 response and then start passing the data.

Security: Data can be encrypted. But if it doesn’t then just use GET.
We used POST in ASP.NET because of the ViewState – just saying “hello” caused the GET data to be filled.
Not limited in length

MVC with Entity Framework Model

Create a new MVC 4 project with a template of Internet Application:

Exercise: changing the default layout:

Add a tag of test in the home controller
Add a tag of test2 in the WhatEver controller

The default layout is configured at _ViewStart.cshtml:

@{
Layout = "~/Views/Shared/_Layout.cshtml";
}

In Razor the file _ViewStart.cshtml determines what is the default layout. You can define a different layout when adding a view:

(what I wrote is the same as leaving it empty, both point to the same file)

The code added will contain the layout specified:

@{
ViewBag.Title = "Test";
Layout = "~/Views/Shared/_Layout.cshtml";
}

<h2>Test</h2>

The code for leaving the text box empty (using the default layout):

@{
ViewBag.Title = "Test";
}

The code for not checking the checkbox (not using a layout):

@{
Layout = null;
}

Self study: Partial views.

Installing Northwinds DB:

Download Northwinds.sql.zip

Using SQL Server Management and a SA account add a new database named: Northwind

Run the script Northwinds.sql using SQL Server Management and the SA account

Under Model folder add ADO.NET Entity Data Model and connect it to the Northwind DB with:

Table: Customers

Table: Orders

Stored Procedure: CustOrderHist

Note: it is advised to separate the Model from the ORM (when there are many-to-one or many-to-many relationships a JSON parser might cause an infinite loop, separating it to simple classes without outside relations solves the problem), but for this demo we won’t.

Reference: AutoMapper that helps mapping between the ORM Model to MVC Model.

Add new controller:

Template: MVC controller with read/write actions TODO using EF

Model: Customer

Data Context NorthwindEntities

Just adding an EF Controller gives out of the box a controller and view for actions such as Index with a list of customers that allows viewing, editing, creating and deleting of rows. Though some of the functionality doesn’t work- for example create.

For example the Index view looks like this:

(clicking on each link passes you to the different views, for example clicking on the first row details the browser redirects to BaseUrl/Customer/Details/ALFKI)

The Create action is divided to two methods:

GET for a blank page(the default is an attribute of HttpGet):

public ActionResult Create()
{
return View();
}

Http Post that after the submit saves the data (it is like in ASP.NET Web Forms PostBack):

[HttpPost]
public ActionResult Create(Customer customer)
{
    if (ModelState.IsValid)
    {
        db.Customers.Add(customer);
        db.SaveChanges();
        return RedirectToAction("Index");
    }

    return View(customer);
}

ModelState.IsValid makes sure that all the fields are correct by checking the data types and more advanced the Data Annotations of the model: one property below another, one is times two another, checking the data against data in the DB… For all of those there will be a nice validation error shown to the user.

Reference Data Validation:

With UI

Returning JSON results:

public JsonResult GetCustomersDetail(string term)
{
    var results = from c in db.Customers
                  where c.CustomerID.StartsWith(term)
                  select new { Id = c.CustomerID, name = c.ContactTitle, other = c.CompanyName};

    return Json(results.ToList(), JsonRequestBehavior.AllowGet);
}

JsonRequestBehaviour enum:

DenyGet – the default option from a security point view, though using Fiddler you can quite easily see the POST request/response.

AllowGet – usually not turning this on causes the system to not work. Just turn it on!

Calling it:

/Customer/GetCustomersDetail?term=a

The results can be made more readable using the Chrome Extension JSONView from Developer Tools:

Note: We can’t use /Customer/GetCustomersDetail/a because we want to pass the term parameter not the id parameter.

JSON Syntax:

[…] array

{…} a type

Instead of = we use :

Commas to separate types/properties

Exercise: Add a text box that creates a combo box of auto complete using a JSON method. Using the event OnChange when 2 seconds pass we will request the data from the server with a GET, receive the data asynchronously, create a type out of the data returned, and set the result in the view as a list items below the TextBox. The only missing information we need is how to make an Ajax call using jQuery.

The JSON method:

public JsonResult GetCustomers(string term)
{
    var results = from c in db.Customers
                  where c.CustomerID.StartsWith(term)
                  select c.CustomerID;

    return Json(results.ToList(), JsonRequestBehavior.AllowGet);
}

In the Documentation we will use the following demo as our base:

$.ajax({
    url: "http://fiddle.jshell.net/favicon.png",
    beforeSend: function (xhr) {
        xhr.overrideMimeType("text/plain; charset=x-user-defined");
    }
}).done(function (data) {
    if (console && console.log) {
        console.log("Sample of data:", data.slice(0, 100));
    }
});

We will change it to:

$(document).ready(function () {
    $("#btnGetCustomers").click(function () {
        var strUrl = "Customer/GetCustomers?term=" + $("#txtCustomer").val();
        $.ajax({
            url: strUrl,
            beforeSend: function (xhr) {
                alert("Good Luck");
            }
        }).done(function (data) {
            alert("Done");
        });
    });
});

The Body looks like:

<input id="txtCustomer" />
<button id="btnGetCustomers">Click Me</button>

Note: The first example from here show using jQuery Ajax with POST. If you want to use the data parameter you must work with a POST. GET doesn’t know how to work with the data parameter.

Now lets debug it to see what the data returns.

Debugging JavaScript:

1. Using IE:

Add break point in VS2012

Set the current browser to IE (on the HTML page right click –> Browse With…):

Choose Internet Explorer and set it as the default browser.

And then just start debugging:

2. Using Chrome F12: Open Sources tab

When you open it it will be empty simply click on this thingy (“Show navigator”):

Choose the file you want to debug:

In our case the HTML page and right click to add break points (you might have to refresh the page).

Now the only thing left as homework is changing the Done alert to display the results as HTML list items below txtCustomer.

jQuery Tip:

$(function() {

is a shortcut to:

$(document).ready(function () {

jQueryUI

Instead of doing all that work we will use jQueryUI for AutoComplete.

You can see at jQueryUI.com an interactive demo of the different Widgets and Effects available at jQueryUI. One of the Widget available is the Autocomplete, you should start by looking at the source code.

The application has only one important line of code – the input tag:

Note: the for="tags" in the label is a HTML5 effect that when the tags is unavailable the label becomes unavailable as well.

Updating our Customers example jQuery code to use jQueryUI:

Step 1: AutoComplete from a constant array (just like the example on the site):

$(function () {
    var customers = [
      "ActionScript",
      "AppleScript",
      "Asp",
      "BASIC",
    ];
    $("#txtCustomer").autocomplete({
        source: customers
    });

Running it with “a” and we get:

Clicking on a list item will cause the text to “Magically” appear in the textbox. But the UI is a bit ugly.

Looking at the tag in Chrome F12, we see that when we used .autocomplete on our textbox jQueryUI changed our HTML to:

class=”ui-autocomplete-input” - if we used a CSS then the autocomplete would have looked nice

autocomplete="off" - jQueryUI uses this variable and set it to “on” when it is working, so that when it is on it “knows” not to work.

When results arrive (for example by entering “a”):

<ul class="ui-autocomplete ui-menu ui-widget ui-widget-content ui-corner-all" role="listbox"
    aria-activedescendant="ui-active-menuitem"
    style="z-index: 1; top: 29.88888931274414px; left: 8.88888931274414px; display: none; width: 136.78472208976746px;">
    <li class="ui-menu-item" role="menuitem">
        <a class="ui-corner-all" tabindex="-1">ActionScript</a>
    </li>
    <li class="ui-menu-item" role="menuitem">
        <a class="ui-corner-all" tabindex="-1">AppleScript</a>
    </li>
    <li class="ui-menu-item" role="menuitem">
        <a class="ui-corner-all" tabindex="-1">Asp</a>
    </li>
    <li class="ui-menu-item" role="menuitem">
        <a class="ui-corner-all" tabindex="-1">BASIC</a>
    </li>
</ul>

To add CSS to the application simply drag Content/themes/base/jQuery-Ui.css above the <script> in <head>. When developing a real application you should use just the css that you need, in this case we could have just used Content\themes\base\jquery.ui.autocomplete.css.

Note: There are tools that analyze your page and give it a score that test you among other things on your CSS usage. For example W3C Validation Service checks for markup validity and helium-css tests for unused CSS classes.

When using the AutoComplete we only used the source property, but there are others which can be viewed at the documentation.

If we wanted an event such as search:

$("#txtCustomer").autocomplete({
    source: customers,
    search: function (event, ui) {
        alert("Go");
    }
});

We can also interfere with the regular function by changing the event (for example if we don’t want to allow the user to search for a certain term):

$("#txtCustomer").autocomplete({
    source: customers,
    search: function (event, ui) {
        event.isDefaultPrevented = true;
    }
});

In the source parameter you can pass:

an array of values: [ "Choice1", "Choice2" ]

an array of objects with value+label pairs: [ { label: "Choice1", value: "value1" }, ... ]

a string url to a service with a parameter of term: SomeUrl?term=X that return an array of values or an array of objects with value+label pairs

a function that returns the data

So using the JSON action we already defined (how lucky was that ):

$(function () {
    $("#txtCustomer").autocomplete({
        source: "Customer/GetCustomers"
    });

And the end result:

Homework:

1. Use a Textbox AutoComplete with a JSON method that returns an array of objects with value+label pairs

2. Textbox with AutoComplete with select event that goes to the server and gets data. Then put that control in the existing view.

3. Two Controllers with a hierarchy: Customer and Orders. Form Order add AutoComplete against Customer.

4. From Customer add AutoComplete of Orders with select event that opens the Customer

ESRI Silverlight MapTip

2013-02-19T02:19:00.000-08:00

Today, while working on a sever bug I have checked to see if the bug doesn’t occur in our Silverlight application as well – it doesn’t. But unfortunately (for me) I found another bug: our MapTip didn’t display some of the components names. After fixing the bug in the server I decided to go after the Silverlight one.

On our previous version our name field was named Name but we had another field named NameEng (for the English name). The MapTip simply displayed the Name field to all elements. On our current version we ported our DB to Oracle and decided to have standardized names such as LOCAL_NAME (previously Name) and NAME (previously NameEng). Hence the problem those component without their name just don’t have a field named LOCAL_NAME – and for some strange reason ESRI doesn’t throw an exception of missing key.

Well I tried searching the web but apparently the feature of a fallback value if the first value is null was only implemented in WPF. I have tried using TargetNullValue (crossing my fingers):

But that just printed [NAME]. I have also tried to use {Binding [NAME]} but that just threw an exception.

I couldn’t change the View Model because the ESRI MapTip works on an inner object in ESRI (which I later found out is an internal sealed class) so adding something like:

was out.

As I see it I had two options:

1. Create a NullToVisibilityConvertor and a NotNullToVisibilityConvertor that when the value is null hides the TextBlock.

public class NullToVisibilityConverter : IValueConverter
{
    #region IValueConverter Members

    public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
    {
        return value == null ? Visibility.Collapsed : Visibility.Visible;
    }

    public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
    {
        return null;
    }
    #endregion
}

(the opposite for NotNullToVisibilityConvertor)

2. Create a Convertor that if one value is null returns the other.

I first tried to use the ConvertorParameter to pass the second value but that came out as simple text. So I have decided to pass the two fields I needed as the ConvertorParameter ('LOCAL_NAME,NAME') and the ESRI class as the binding value:

Now what I found (through Debugging and JetBrains dotPeek) is that though ESRI passes an internal sealed class named ObservableDictionary (and I cannot cast to it in my convertor) that class implements IDictionary<string, object> so I can just cast the value to that:

public class DictionaryValuePickerConverter : IValueConverter
{
    public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
    {
        var dictionary = value as IDictionary<string, object>;
        var stringPrameters = parameter as string;

        if (dictionary == null || stringPrameters == null)
        {
            return String.Empty;
        }
        var parameters = stringPrameters.Split(",".ToCharArray());

        foreach (var key in parameters)
        {
            if (dictionary.ContainsKey(key))
                return dictionary[key];
        }
        return String.Empty;
    }

    public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
    {
        throw new NotImplementedException();
    }
}

I actually hate both options and had to ask my team leader what he thought. At the end we decided to go with option 2, if somewhere down the line we will need (and God help us if we do) another fallback name field we can just add another using “,YET_ANOTHER_NAME” and it will work…

I hope no one needs this…

HTML5+MVC Course, Day 1

2013-02-17T10:03:00.001-08:00

My company is having a HTML5 and MVC course by E4D. As is my habit I like to write things down – for review and prosperity…

The course is presented by Yair Siwek.

The first day was more an overview on HTML5, JavaScript and quick uses of jQuery.

Overview

The course started with an overview about what we are going to go over:

JavaScript is the language we use in the Browser, it advantage is in the fact that it is text so it passes the Firewall. Firewalls hate everyone, but text somehow passes…

JavaScript gives events on: the DOM, button click, data loaded, etc.

But it’s script that might cause Memory Leaks.

jQuery is a JavaScript library that started as a simple selector library to search for elements in the DOM tree. Then you can change those elements: text, CSS classes, events, etc. A lot of inline code, anonymous types/functions.

jQueryUI is a JavaScript library that uses jQuery as it’s base and adds gadget code like AutoComplete text box, calendar, tabs, accordion, drag & drop, etc.

The problem in jQueryUI is that the process of creating a new control is very long (gets out once per year), so other people started creating open source plugins in GitHub.

Knockout is a MVVM framework for JavaScript. There are other frameworks that are better for other usages (such as Backbone, Angular, Ember…).

The mobile world is different. For example in Facebook they have a Java (Android) team, Objective C (IOS) team and HTML5 for all the rest.

Tools:

Visual Studio 2012 – has intellisense for HTML5

With NuGet installed: insures all the dependencies and references are working together. Can be set so that the source of the files sits inside the organization.

Browsers: Chrome, Firefox, IE9

MVC

MVC = Model View Controller

The course is going to use Controller and (maybe) Model from MVC and the view in the client will be receiving JSON data from MVC.

Why not use WCF?

WCF uses SOAP which include a lot of metadata in the header (like security, transaction, reliable messaging, etc. ).

The browsers allow xml of type POX (Plain Old Xml) but not SOAP.

Microsoft implemented WCF-Ajax that returns JSON but that causes Cross Side Scripting problems. There is a pipeline in WCF that in the end cuts all the data to JSON – you are doing a lot of work but in the end just take a pips of the data. JSONP solved the problem but not all the controls know how to work with it.

The final point: it’s too much work when MVC is so much simpler.

HTML games

Open a new ASP.NET MVC4 project

Open a new HTML page

<!DOCTYPE html>

The tag means this is HTML5

Can be and should be cleaned up to just <html>.

In Chrome – F12 for Developer Tools, the best tool to play with the HTML and CSS of the page.

Under Styles are the CSS values we specified in the page.

Under Computed Style are the actual values in the browser.

For example: font-size: 1.5em is 1.5 of the default font size in the operating system, but in the browser that value means 24px.

Options of writing styles:

1. inline

<p style="font-family:Aharoni">Bad inline: Hello World2</p>

2. header

<head>
    <style>
        .MyClass1
        {
            background-color:gray;
            color:white;
            font-size:1.5em
        }

        p.MyClass1
        {
            background-color:gray;
            color:yellow;
            font-size:1.5em
        }
    </style>
</head>

Got the class MyClass1

<p class="MyClass1">Hello World1</p>

Got the class p.MyClass1. Because it was explicitly created for <p> tags it is stronger than just MyClass1.

<p class="MyClass1 MyClass2">Hello World3</p>

Got both MyClass1 and MyClass2 styles.

ID specific styles

#pId
{
background-color:maroon
}

Only relevant for the tag with the id pId:

3. Style for outside CSS file:

Using it is the same as using header styles.

Rule: inline style is stronger than header style who in turn stronger than an outside CSS file.

Solution order conventions:

Content folder for application styles.

Scripts folder already contains NuGet scripts.

Scripts\App folder for application scripts

Demo Core JavaScript: Playing with the Page’s Layout

From 3 DIVs

<body>
    <div id="Div1">header</div>
    <div id="Div2">left</div>
    <div id="Div3">main</div>
</body>

To (just using styles):

No white in the margins only have the 3 colors.
Header should be 40px in height
Left should be 20% of the width and Main should be the other 80%
Left and Main should have all the rest of the screen

The next step is for the Header and the Left not to move when scrolling.

In Chrome the body tag has a default margin of 8px, to have no white in the page we need to change that to a margin of 0px. In other browsers the defaults might be different.

To make Main be on the right of left and not be a line below it we need to use the float style.

The end result is:

Demo in Core JavaScript: playing with z-index

One DIV is above the other, events such as on click for div2 will not fire when clicked on the area that div1 takes.

If we want to change the z-index on click then we have several options of writing methods:

1. The simplest option:

onclick="Div2_OnClick()"

2. For if we want to escape the function with return false:

onclick="return Div1_OnClick()"

3. With a timer (only mentioned not shown)

The script Tag should be after the the style tag if you want to use styles in the script.

The setting of the zIndex in the WindowOnLoad method is an alternative way to set the styles but should not be used. The body onload will only fire when all the page loads – that includes downloading all the images.

Demo Hello jQuery

Showing manipulation of the following HTML:

<body>
    <button id="btn1">Click me</button>
    <div class="MyClass" id="div1">this is div1</div>
    <div id="div2">
        this is div2
        <div class="MyClass" id="div3">
            this is div3
        </div>
        <div class="MyClass" id="div4">
            this is div4
        </div>
    </div>
</body>

Adding the jQuery library by drag and drop from the Scripts folder to the <head> tag (after getting it from NuGet or from the jQuery site), or by writing:

When working with jQuery you have the option of working with:

The min version (jquery-#.#.#.min.js) which weights a lot less, has no documentation and better performance.
The regular version (jquery-#.#.#.js) which has documentation and is generally better for debug.
The intellisense version (jquery-#.#.#.intellisense.js) for people working with the min version , they can add intellisense using this file (if you are working with the regular version you don’t need the intellisense version).

jQuery has several selector options (i.e. $() ):

Using object: $(document) returns the DOM object
Tag name: $(‘div’) selects all the DIV tags
Tag ID: $'(‘#Tag_Id’) selects the tag that has the Id Tag_Id
Tag Class: $(‘.Tag_Class’) selects all the tags that use the class Tag_Class

There are more selectors which can be read here.

jQuery has an event for OnLoad minus, for when the DOM has loaded but the images haven’t:

All the operation in jQuery are done with functions. The event ready receive an anonymous function that does the action (you can also write a regular JavaScript function but it is usually done with Anonymous types):

Set the value into html:

$("#div3").html("can you see me?")

Returns the html text:

$("#div3").html()

The previous example of:

div1.style.zIndex = 102;

Is bad code because it might not work in all browsers, the right code should be:

document.getElementById("div1").style.zIndex = 102;

In jQuery we can simply do $(“div1”) and it will work in all browsers because jQuery insures it.

//bad code (will go over the tree a few times to get div3):
$("#div3").addClass("redStyle");
$("#div3").html("can you see me?");
alert($("#div3").html());

If you open debugger and step into all the $(“#div3”) you will see that jQuery iterates through the DOM tree for each call of $(“#div3”) and after each action returns the original selection (i.e. Fluent Interface).

The good ways:

Using a temporary variable:

var $div3 = $("#div3");
$div3.addClass("redStyle");
$div3.html("can you see me?");

Using the variable name $div3 is a jQuery convention of saying this variable is from jQuery.

Better way of doing everything inline:

$("#div3")
.addClass("redStyle")
.html("can you see me?");

Demo – Change Class

$("#btn1").click(function() {
    $("#div3")
        .removeClass("redStyle")
        .addClass("boldStyle")

But on the next click nothing will happen, another option is using toggle (which uses the current state):

$("#btn1").click(function() {
    $("#div3")
        .toggleClass("boldStyle")
        .toggleClass("redStyle")

Adding a style to a given class (if the class already contained a color than this definition will override it):

$(".MyClass")
.css("color", "red");

Hiding an element slowly and in the end change the text of the button to “Done”:

$("#div4").toggle("slow", function() {
$("#btn1").html("Done");
});

jQuery behind the scenes changes the style of the display to none (and if clicked again to block).

Multiple Selectors:

$(".MyClass, #div2")
.css("font-family", "serif");

All the tags that their Id is div2 or have the class MyClass.

Ancestor descendant:

$("#div2 .MyClass")
.css("color", "blue");

All the tags that their ancestor is div2 and have the class MyClass.

And:

$(".MyClass.redStyle")
.css("color", "purple");

All the tags that have the both class MyClass and redStyle.

Resources:

jQuery, and API Documentation

The 30 CSS Selectors you Must Memorize

jQuery Fundamentals

W3School CSS3 – not very useful

VS2010: “The project type is not supported by this installation.”

2012-11-08T09:48:00.000-08:00

Before I found the extension for Windows Live Writer – Paste as Visual Studio code I had (for 3 days) used Code4Blog Extension but I was dismayed by it’s bugs (I opened some of them). I had this draft lying around and decided to just post it because it seems many people suffer from this exception.

In the extension’s page the author wrote:

“Code4Blog is now published on CodePlex. You are welcome to work with the source code directly!”

So I have decided to do just that.

Opening the solution I got the following error in the output window:

C:\…\Code4Blog\Code4Blog\Code4Blog.csproj :

error : The project file 'C:\…\Code4Blog\Code4Blog\Code4Blog.csproj' cannot be opened.

The project type is not supported by this installation.

So I kinda guessed it has to do with a project for extending VS. But why doesn’t VS2010 give me what project type is missing?

Looking inside the project file (good thing that the project was down ):

I found the project type here:

 <ProjectTypeGuids> {82b43b9b-a64c-4715-b499-d71e9ca2bd60};

                    {60dc8134-eba5-43b8-bcc9-bb4bc16c2548};

                    {FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}

</ProjectTypeGuids>

Searching for the first Guid on the list, I found this forum. From there I got to the Extending Visual Studio page and from there to the Visual Studio 2010 SDK. After installing that (you should close VS even though it doesn’t ask you to, because otherwise you won’t be able to open the project) the project worked.

I later found this site with a list of Guid and what they mean some of it is still relevant for VS2010. The only Guid not there was the first I searched for (damn you Murphy) {82b43b9b-a64c-4715-b499-d71e9ca2bd60} (the others mean WPF project and C# project respectively).

Well it might have worked but building failed because one of the resources didn’t work (icon) and changing it to another caused my VS to hang on the build – maybe I should have uninstalled the extension first? Killed the devenv, uninstalled the extension and the build passed.

In the end I decided the hassle was just to great and just used “Paste as Visual Studio code”. If you want to read about all the options I went through you can read them here.

Resources:

Code4Blog Extension page

Code4Blog CodePlex page

Extending Visual Studio page

Visual Studio 2010 SDK

VS2010 Project Type Guids

Silverlight: TextBox binding only works when control not in focus

2012-05-31T02:36:00.000-07:00

Yesterday I got to the point that most of my priority one tasks were done ahead of schedule so I have decided to go back to old controls and rewrite them to work with the MVVM pattern. I even got to an old argument with my co-worker Felix about using code behind for UI purposes (he is a firm believer in Expression Blend but for implementing Watermarks he decided to use code behind – I refactored it using this).

So after fixing the Watermark and the AutoCompleteBox that used a service to populate the items in the code behind (using this) I got down to the actual minor functionality tweaks. Like when the user presses enter in AutoCompleteBox or TextBox some Action is triggered. For the AutoCompleteBox I used the Attached Property Approach (I changed it a bit to be for KeyUp instead of KeyDown) just without the CommandArgument since the selected item was already with Data Binding. And it worked just fine.

But with TextBox this approach just didn’t work – because TextBox binds the data only when it is out of focus!

I found some more people with the same problem:

Go4Answers: TextBox TextChanged event doesn't cause data binding to update

Silverlight Forum: TextBox TextChanged event doesn't cause data binding to...

They used the behavior solution (which while browsing for solutions I missed (I guess I am turning lazy because in StackOverflow I would have gotten the right answer in the first page/answer)). I just decided to use the CommandArgument from Attached Property Approach (the same one I didn’t need for AutoCompleteBox):

The View Model:

public RelayCommand<string> EnterSomeTextCommand
{
get;
private set;
}

(using GalaSoft MvvmLight)

Now you can just use the value.

Update:

@yesez5 in the comments have added his blog post (in Spanish, or in English (using Google Translate)) about using a built-in behavior from the Prism Library (that way you don’t even have to add code implementations for Action triggers or Behaviors in your project).

Resources:

IBlog<Johan>: Silverlight Watermark TextBox Behavior

Peter Kuhn: Using the AutoCompleteBox

StackOverflow: Invoke Command When “ENTER” Key Is Pressed In XAML (and Answer)

Go4Answers: TextBox TextChanged event doesn't cause data binding to update

Silverlight Forum: TextBox TextChanged event doesn't cause data binding to...

Zoltan's Dev Corner: Binding update on TextBox.TextChanged event using Behaviors

GalaSoft MvvmLight

Lightside: Update TextBox binding when the value changes

Default Browsers bug in VS2010

2012-05-20T11:00:00.001-07:00

If you don't know (or haven't encountered it yet) there is a bug in VS that causes the settings of the default browser to not save.

For example: I am using Firefox as my default Windows browser but prefer debugging using IE (mainly because if IE dies my browser is still alive). And I have to change the browser much too often.

There isn't a fix for this but there is a workaround:

From VS -> Tools -> Extension Manager

-> Online Gallery -> Search:

WoVS Default Browser Switcher

Download and install the extension and when prompt restart VS.

Now in VS: View -> Toolbar -> enable: Default Browser Switcher

And you will get:

(image from WoVS Default Browser Switcher extension page)

And you will see what default browser is turned on.

I prefer putting it just below the debug toolbar so that it will be in front of my eyes but you can place it where ever you may like…

There is also a context menu for viewing web pages but it is not as useful as the toolbar.

You can read more about it here.

Resources:

Microsoft connect: VS2010 forgets default browser settings

Extension: WoVS Default Browser Switcher

A day without the Debugger

2012-05-09T06:16:00.000-07:00

On Sunday morning I found out the Visual Studio 2010 Debugger has stopped working for me. It wasn’t isolated to a single project or technology. Attach to process didn’t work in Silverlight and running Unit Tests in debug mode gave the usual “The breakpoint will not be hit. No symbols have been loaded for this document.”:

Visual Studio was in Debug mode just didn’t pass through any breakpoint (just to check that the Unit Tests used the latest build I added Assert.Fail and the Unit Test failed).

So, first I thought to give myself a challenge and try to work without a debugger. All was of the good until I had a small and stupid bug that I was sure I could just solve using Unit Tests, but the server side worked and I could see the data passing from the client using Fiddler. It didn’t help that in the front of my mind I just knew that with a Debugger this will be solved in minutes. So I dropped the challenge and started investigating the problem.

First I went through the usual suspects:

Made sure I was building in debug mode (Solution Configuration toolbar –> Debug)
Deleted the Bin\Debug files and rebuilt
Changed the Build Platform from “Any CPU” to x86
Made sure the project build’s debug info was on full (Project properties –> Build –> Advanced… –> Debug Info should be full):

Restarted the computer
Enabled Just My Code (Options-> Debugging –> General), since I like having symbol debugging it was turned off
Reset the setting using Devenv.exe /resetsettings
I decided against using Devenv.exe /SafeMode (but next time I will use it)

I was very close to trying the repair on the windows Control Panel. But then I noticed that while the debugger didn’t break on any breakpoint it did show a few BadImageFormatException exceptions being thrown, setting Visual Studio to break on those exceptions (Debug menu –> Exceptions… (or Ctrl+D+E) –> Find: “BadImageFormatException” and checking the thrown:

Running it again:

System.BadImageFormatException occurred
Message=Could not load file or assembly 'dbghelp.dll' or one of its dependencies. The module was expected to contain an assembly manifest.
Source=mscorlib
FileName=dbghelp.dll
FusionLog=""
StackTrace:
       at System.Reflection.AssemblyName.nGetFileInformation(String s)
       at System.Reflection.AssemblyName.GetAssemblyName(String assemblyFile)
InnerException: System.BadImageFormatException
       Message=Could not load file or assembly 'dbghelp.dll' or one of its dependencies. The module was expected to contain an assembly manifest.
       FileName=dbghelp.dll
       FusionLog=""
       InnerException:

Or just the Dlls:

dbghelp.dll
DDConfig.dll
LanguagePackage.dll
Microsoft.VisualStudio.Coverage.Symbols.dll
Microsoft.VisualStudio.ProgressionIcons.dll
Microsoft.VisualStudio.QualityTools.RecorderBarBHO100.dll
pidgenx.dll
SqlTDiagN.dll
SqlToolsVSNativeHelpers.dll
symsrv.dll

Searching the file system for one of them dbghelp.dll,the icon for the DLLs was for dotPeek which got me thinking about uninstalling JetBrains early access tools dotPeek and dotCover (after closing VS) – just to be sure it wasn’t them…

Trying Debug again and everything worked.

The weird part is trying to reinstall those apps (with the exact same MSIs) didn’t reproduce the problem. Looking back I remember restarting my computer before all this started but even restart won’t reproduce the problem.

Another case for the “computer's nondeterministic nature”…

Resources:

Peter Gfader 's .NET Blog: VS2010 - Debugging not working

StackOverflow: debugging is not working in my visual studio 2010

MSDN Forums: Debugger not working on Visual Studio 2010

COMException: Abstract Data Types not supported

2012-04-11T06:18:00.000-07:00

“Full” Exception:

System.Runtime.InteropServices.COMException: Abstract Data Types not supported [SHEMA.LAYER][STATE_ID = 0]

The real problem:

When calling:

workspace.OpenFeatureClass(layerName)

(where workspace is IFeatureWorkspace)

with a layer that was not defined in the SDE. You will get the given error.

Why did it happen to me?

When we moved to Oracle we decided to create our tables using plain SQL and define our shape columns as SDO_GEOMETRY. This was done so other teams working on our DB on their computer won’t need to install SDE and could run Unit Tests using non geography data. Obviously we forgot to run the register command on one of our tables and I got this lovely exception…

The fix:

Registering the layer using:

sdelayer -o register –s SERVER –u DB p PASSWORD –i SDE_INSTANCE -t SDO_GEOMETRY -e p –l LAYER,shape -R 2 -C OBJECTID,SDE

(in red what you must change, blue should be according to your DB).

One ESRI exception mapped, an ∞ to go…

Selecting Distinct Rows

2012-03-04T08:54:00.001-08:00

Or how the definition of the request can get us confused?

One of my team members asked me today to help her in a SQL query and although I hate SQL I tried to help her. She wanted to select a row by a distinct field – ColumnA and to view other columns in that row, so I started the query with the distinct keyword:

SELECT DISTINCT ColumnA
FROM SomeTable

But run into the problem that she wanted those other columns of the row which weren’t distinct.

So I Googled Distinct and got lost. After taking a coffee break I changed my search to “sql select id of first distinct” the first link was of StackOverFlow: SQL SELECT DISTINCT ID from copies ORDER BY CID

Which told me of course that I was using the wrong keyword and should have used the Group by keyword.

The end result query:

SELECT *
FROM SomeTable
WHERE OBJECTID IN ( SELECT MIN(OBJECTID)
FROM SomeTable
GROUP BY ColumnA)

Was what she wanted.

God bless coffee (and StackOverFlow)…

Resources:

StackOverFlow - SQL SELECT DISTINCT ID from copies ORDER BY CID

SQL Server 2008: Forget a user/password

2012-03-03T13:17:00.000-08:00

For a long time now I am stuck with a saved user in SQL Server 2008 that I just didn’t want to use:

Once upon a time I used that user (that has almost no permissions) for just one action – unfortunately I checked the “Remember password” and whenever I choose that server in the servers combo box I got that damn user. Well I finally had enough!

So I Googled “remove sql server 2008 user password Remember password” and found this StackOverFlow post.

The solution is fairly easy, just rename: C:\Users\%username%\AppData\Roaming\Microsoft\Microsoft SQL Server\100\Tools\Shell\SqlStudio.bin to SqlStudio.bin0 (as a rule I never delete a file I don’t have a backup to). And now SQL Server doesn’t remember anything (not even the servers I was on):

But that is just a hundred times better than the previous headache…

Resources:

StackOverFlow: Removing the remembered login and password list in SQL Management Studio

PowerShell: Event Viewer saver

2012-02-09T10:34:00.000-08:00

Two weeks ago I was helping out at my old company (it is part of my old contract with them). When one of my friends there told me they just can’t look at the Event Viewer anymore. Why? Well it seems another friend got a little overboard in logging stuff and the Event Viewer turned from logging Errors and important events to “real time logging” (their words). So being the helpful guy that I am I told them about PowerShell. I even pointed them to some blogs in the web.

Today I decided to follow my own advice and check out PowerShell (and maybe write something they could use). So I went through some basic tutorials (1, 2, 3 reading briefly, 4, 5, 6 (can be skipped), 7, 8, 9, there are more tutorials but they are not needed for this simple task). When following tutorials I sometimes like to use my own example, that way I can be sure I am understanding things like a human and not like a monkey (“Monkey see, monkey do”).

On the second tutorial I already knew I had to use ‘get-eventlog’ or the newer ‘get-winevent’ command (by using ‘get-command -Verb Get’ and looking at the options). By looking at the help for these commands (by using ‘get-help get-eventlog’) I found out that get-winevent needs newer operating systems (minimum Windows Vista) which are not available to my friends…

So what do I want my script to do?

1. Display only the errors from Event Viewer’s Application log (and filter by the application name).

Looking at the examples was enough for this:

get-eventlog -logname System -EntryType Error

Wanting something more advanced (I learned that you shouldn’t use TimeWritten in the where since it is a string object, see sorting below):

Get-Eventlog Application -After ([datetime]::Parse("2012-02-08 17:00")) | where {($_.entryType -eq "Error")}

This way you can “add” warnings not just errors:

Get-Eventlog Application -After ([datetime]::Parse("2012-02-08 17:00")) | where {(($_.entryType -eq "Error") -or ($_.entryType -eq "Warning"))}

Adding sorting to view the errors from old to new, added difficulty because TimeWritten is a string with a problematic format (M/d/yyyy h:m:s AM/PM) that doesn’t work with string comparison:

Get-Eventlog Application -After ([datetime]::Parse("2012-02-08 17:00")) | where {($_.entryType -eq "Error")} | select EntryType,InstanceId,Message,Category,Source, @{Name="TimeWritten";Expression={ ([datetime]::Parse($_.TimeWritten)).ToString("yyyy-MM-dd HH:mm:ss") }} | sort -Property TimeWritten

Wanting to actually see the message (since PowerShell by default limits the column size):

Get-Eventlog Application -After ([datetime]::Parse("2012-02-08 17:00")) | where {($_.entryType -eq "Error")} | select EntryType,InstanceId,Message,Category,Source, @{Name="TimeWritten";Expression={ ([datetime]::Parse($_.TimeWritten)).ToString("yyyy-MM-dd HH:mm:ss") }} | Format-list -Property EntryType, Source, Message, TimeWritten

2. Export the errors from 1 to a csv file (so that the errors will become less real time and more all time).

As seen in the bottom of tutorial 5:

Get-Eventlog Application -After ([datetime]::Parse("2012-02-08 17:00")) | where {($_.entryType -eq "Error")} | select EntryType,InstanceId,Message,Category,Source, @{Name="TimeWritten";Expression={ ([datetime]::Parse($_.TimeWritten)).ToString("yyyy-MM-dd HH:mm:ss") }} | Export-CSV "EventErrors.csv"

3. Do an infinite loop that does 2 on new errors, so that errors in the night shift will be stored.

That was a bit more complex simply because I wanted to append rows to the CSV.

$currentTime = get-date
$currentDate = get-date -format "yyyy_MM_dd"
if(test-path "EventErrors_$currentDate.csv")
{
$record = Import-CSV "EventErrors_$currentDate.csv" | sort -Property TimeWritten | Select-Object -Last 1 | select TimeWritten
$currentTime = [datetime]::Parse($record.TimeWritten)
}
while($true)
{
$currentDate = get-date -format "yyyy_MM_dd"
$data = Get-Eventlog Application -After $currentTime | where {($_.entryType -eq "Error")} | select EntryType,InstanceId,Message,Category,Source, @{Name="TimeWritten";Expression={ ([datetime]::Parse($_.TimeWritten)).ToString("yyyy-MM-dd HH:mm:ss") }}
$currentTime = get-date
if($data -ne $NULL)
{
if(test-path "EventErrors_$currentDate.csv")
{
"File EventErrors_$currentDate.csv exists, importing data."
$data = (Import-CSV "EventErrors_$currentDate.csv") + $data | sort -Property TimeWritten
}
$data | Export-CSV "EventErrors_$currentDate.csv"
}
Start-Sleep -s 20
"Ping $currentTime"
}

Notes:

The script won’t work if the file is opened in Excel (but the csv can be copied or opened with notepad).
You can adjust the sleep time, I choose 20 seconds.
Since it’s an infinite loop it can only be exited by Ctrl+C

Output in Excel:

By the way an advanced script to this could be outputting the errors to the DB… But that can be considered a version in some companies.

Resources:

PowerShell Tutorial 1: Configuring the PowerShell Console

PowerShell Tutorial 2: PowerShell Commands – Cmdlet

PowerShell Tutorial 3: PowerShell Aliases

PowerShell Tutorial 4: Using Cmdlet Options

PowerShell Tutorial 5: Windows PowerShell Providers

PowerShell Tutorial 7: Accumulate, Recall, and Modify Data

PowerShell Tutorial 8: Conditional Logic (if, elseif, else, and switch)

PowerShell Tutorial 9: Getting Loopy

Silverlight Windows Authentication (the Prism way)

2012-01-25T10:51:00.001-08:00

Note: this was done with Silverlight 5 and Prism 4

In the past month our team started a new project and after many discussions of how to build it right, decided to use the Prism framework (you an read more on Prism here). I was put in charge of providing a service (as in a Silverlight utility class) for Authentication and Authorization using Windows Authentication and Active Directory groups.

The way we handled that in our previous project was using the built in services in RIA services (the only use we had for RIA Services). But that implementation used:

Code in the main Silverlight Application project (not in Class Library projects)
Configuration in App.Xaml
Configuration in web.config

So I tried for two days to create the code in a Silverlight Application project that is not the main project (not the one referenced in the ASPX page), with no success and many errors. I found many forum posts that claim it is possible but all the examples were either Forms Authentication or used the main Silverlight Application (like this one or this one). The main problem I found is the configuration that should be put in the App.Xaml:

<Application.ApplicationLifetimeObjects>
<app:WebContext>
<app:WebContext.Authentication>
<appsvc:WindowsAuthentication/>
</app:WebContext.Authentication>
</app:WebContext>
</Application.ApplicationLifetimeObjects>

The app preface is from the generated code that is generated in a Module Silverlight Application, but the App.Xaml is in the Main Silverlight Application and it just doesn’t work to use it outside of the project it was generated at. Adding App.xaml in the Module Silverlight Application won’t work either because Prism doesn’t call the App.Xaml of inner Modules (though having that Module run as it’s own main application does work).

In the end I decided to try to implement the RIA Authentication myself using WCF Services (me and my team leader have burned to much time on RIA by that point… and since we use it for just authentication it didn’t seem justified).

The WCF Services web site must be configured in IIS to use Windows Authentication. The code for the service is:

[ServiceContract]
public interface IAuthenticationService
{
[OperationContract]
User GetUser();
}

[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
public class AuthenticationService : IAuthenticationService
{
public User GetUser()
{
var result = new User { Name = OperationContext.Current.ServiceSecurityContext.WindowsIdentity.Name };
if (OperationContext.Current.ServiceSecurityContext.WindowsIdentity.Groups != null)
{
result.Groups = new List<string>();
foreach (var userGroup in OperationContext.Current.ServiceSecurityContext.WindowsIdentity.Groups)
{
result.Groups.Add(userGroup.Translate(typeof(NTAccount)).Value);
}
}
return result;
}
}

The user groups default values are in SID format so the code:

userGroup.Translate(typeof(NTAccount)).Value

translate it to readable text.

User is defined as:

[DataContract]
public class User
{
[DataMember]
public string Name { get; set; }
[DataMember]
public IList<string> Groups { get; set; }
}

Web.Config code:

<configuration>
<system.web>
<compilation debug="true" targetFramework="4.0" />
<authentication mode="Windows" />
</system.web>
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="AllServicesBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="true" />
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
<behavior name="">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="false" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<customBinding>
<binding name="customBinaryEncodedBinding">
<binaryMessageEncoding/>
<httpTransport authenticationScheme="Negotiate"/>
</binding>
</customBinding>
</bindings>
<services>
<service behaviorConfiguration="AllServicesBehavior" name="DllShepherd.AuthenticationService">
<endpoint address="" binding="customBinding" bindingConfiguration="customBinaryEncodedBinding" contract="DllShepherd.IAuthenticationService"/>
</service>
</services>
</system.serviceModel>
</configuration>

The important part is:

<httpTransport authenticationScheme="Negotiate"/>

and:

<authentication mode="Windows" />

that enables the authentication and binaryMessageEncoding for Silverlight.

In the Silverlight Services Class Library project (Services is part of the Prism terminology for client services, usually Singleton utility classes), add the service reference (I do this part usually in code) and consume it using usual Silverlight code:

public string UserName { get; private set; }
public event EventHandler UserLoaded;
public void GetUser()
{
Client.Begin("GetUser", GetUserCompleted, null, null);
}
private void GetUserCompleted(object sender, ObjectClient<DllShepherd.IAuthenticationService>.ClientEventArgs e)
{
Dispatcher.BeginInvoke(() =>
{
var result = e.LoadResult<User>();
UserName = result.Name;
AuthorizationService.Roles = result.Groups;
if (UserLoaded != null)
UserLoaded(this, null);
});
}

AuthorizationService will later use the Groups to check if the user is authorized to view Modules, Views or data.

In the Prism way the Modules now know only the interface for this Silverlight service (the interface is in the Infrastructure project which everyone knows), which look something like:

public interface IAuthenticationService
{
#region EVENTS
event EventHandler UserLoaded;
#endregion
#region METHODS
void GetUser();
#endregion
string UserName { get; }
}

But the implementation is only known by the Application (not the Modules which just use IAuthenticationService with a Prism Import) and there is no special code or configuration needed in the Application.

Resources:

Silverlight.Net Forum - Prism, RIA & Authentication (VS 2010)

Silverlight.Net Forum - PRISM 4 and RIA services

Prism for Silverlight/MEF in Easy Samples. Part 1 - Prism Modules (there are three parts)

ArcObjects: TOC

2011-12-26T03:35:00.002-08:00

List of ArcObjects posts:

Introduction
Getting Started
Extending the Framework
Workspace
Workspace Provider
Workspace is Down
Licensing
Com Releaser
Cursors and IsRecycling
WorkspaceUtils
Multithreading
Mocking

IceRocket Tags: ArcObjects

ArcObjects: Workspace is Down

2011-12-26T03:35:00.001-08:00

This post is the sixth post in my ArcObjects series.

This is a direct continuation from my post ArcObjects: Workspace Provider.

Back in March our DB was down over the night and unlike all the other systems ours didn't just resume work after the DB was brought up, it needed the team’s involvement.

The Exception we got was:

System.Runtime.InteropServices.COMException was unhandled

HelpLink=esri_csGeoDatabase.hlp

Message=Failure to access the DBMS server [Microsoft SQL Server Native Client 10.0: Communication link failure] [SERVER.SCHEMA.LAYER_NAME]

Source=esriDataSourcesGDB.SdeWorkspace.1

ErrorCode=-2147155559

StackTrace:

at ESRI.ArcGIS.Geodatabase.IFeatureClass.Search(IQueryFilter filter, Boolean Recycling)

InnerException:

So I recreated the error by restarting the "SQL Server (MSSQLSERVER)" service in our DB with some special code.

Symptoms:

1. The IWorkspace status:

((IWorkspace) _workspace).Exists();

Returns true

2. Then I started looking at other places like ESRI’s diagrams. One of the diagrams:

(Not mine EDN's)

IWorkspaceStatus sounded a bit off to me that it is under WorkspaceFactory and not under Workspace but the real problem was what was written in the documentation. There are no implementations of this according to EDN…

But I found another interface IWorkspaceFactoryStatus, managed to point it to my workspace and got back:

((ESRI.ArcGIS.Geodatabase.IWorkspaceFactoryStatus )workspaceFactoryTemp).PingWorkspaceStatus((ESRI.ArcGIS.Geodatabase.IWorkspace)_workspace)

{System.__ComObject}

[System.__ComObject]: {System.__ComObject}

ConnectionStatus: esriWCSAvailable

Workspace: {System.__ComObject}

I just thought to myself how can it possibly be Available, until I look at the documentation at esriWorkspaceConnectionStatus:

Available doesn't mean it's up - just that it was lost and now is available. And I was sure it meant something else… (Couldn’t they name it something like LostAndNowAvailable or AvailableToReconnect? – Available just does not cut it!!!).

The end result looks like this:

if (!IsWorkspaceRunning(_workspace))
{
ReactivateWorkspace();
InitializeWorkspaceUtils(_workspace);
}
return _workspaceUtils;

Checking if the workspace is running:

private bool IsWorkspaceRunning(IWorkspace workspace)
{
IWorkspaceFactoryStatus workspaceFactory = new SdeWorkspaceFactoryClass();
return workspaceFactory.PingWorkspaceStatus(workspace).ConnectionStatus == esriWorkspaceConnectionStatus.esriWCSUp && workspace.Exists();
}

If it’s down recreate it (and if it can’t be recreated throw an exception):

private void ReactivateWorkspace()
{
IWorkspaceFactoryStatus workspaceFactory = new SdeWorkspaceFactoryClass();
var status = workspaceFactory.PingWorkspaceStatus(_workspace);
if (status.ConnectionStatus != esriWorkspaceConnectionStatus.esriWCSDown)
{
_workspace = workspaceFactory.OpenAvailableWorkspace(status);
}
//try to create it again:
_workspace = CreateWorkspace(_connectionString);
status = workspaceFactory.PingWorkspaceStatus(_workspace);
if (status.ConnectionStatus == esriWorkspaceConnectionStatus.esriWCSDown)
{
throw new ApplicationException("The Connection to the database was lost. Please contact your system administrators.");
}
}

You can also look at the full code here.

Hopefully next time the DB is down it will just recreate IWorksapce…

Resources:

The forum post I wrote

ArcObjects: Workspace Provider

2011-12-26T01:28:00.001-08:00

This post is the fifth post in my ArcObjects series.

As I mentioned in my previous post you cannot create a workspace you have to receive it from some place. That place is called the WorkspaceFactory (or as usual IWorkspaceFactory). Now since getting the workspace is useful and usually only done once we put it in a singleton class named WorkspaceProvider.

As can be seen in the IWorkspaceFactory documentation there are many implementation of this interface. The implementation differ in the Space where the workspace is doing it’s work:

SdeWorkspaceFactory – works on a Sde DB
CadWorkspaceFactory – works on CAD files
AccessWorkspaceFactory and FileGDBWorkspaceFactory – work on File based Geodatabases

These classes provide the interface to connect to the Space the same way ArcDesktop tools do so by the Open method. You can either use a file containing the connection (created in ArcCatalog), use IPropertySet a list containing the connection information (see here for details) or by using connection string (the other options just use this option behind the scene).

Sde connection string are separated to two categories (more details in one of my previous posts: ArcSDE: Connection to the Geodatabase):

Regular connection: "SERVER={0};INSTANCE={4};USER={2};PASSWORD={3};Database={1};VERSION=SDE.Default"
Direct connection: "SERVER={0};INSTANCE=sde:{5}:{0};USER={2};PASSWORD={3};Database={1};VERSION=SDE.Default"

Where:

{0} – Server name
{1} – Database name
{2} – User name
{3} – password
{4} – Sde service number
{5} – Type of the server (i.e. sqlserver)
If you need to you can also add the Sde version

As you can see direct connection doesn’t use the Sde service number, that is because it doesn’t use the Sde service (it does all the work at the client). Most times you won’t need to change the Server type so that’s out as well. My code only uses Direct Connection string (since ESRI recommends using it).

In my experience most applications will use only one IWorkspace – the one connecting them to the application Sde. For that reason WorkspaceProvider uses DefaultDbConfigSection which is both a Configuration Section and a Connection String builder (I use it for both the Sde and regular DB Connection string). To prevent unnecessary creation of IWorkspace objects WorkspaceProvider will also store the last IWorkspace created and the connection string used to create it.

As it is the only public methods needed for this class are:

1. Using Sde Connection String from web/app.config file:

public WorkspaceUtils GetWorkspace()

2. Using Sde Connection String provided:

public WorkspaceUtils GetWorkspace(string sdeConnectionString)

3. Using a Cad file:

public CadWorkspaceUtils GetCadWorkspace(string path)

4. Using either GDB or MDB file paths:

public FileWorkspaceUtils GetFileWorkspace(string filePath)

The full code can be found here.

Most of the code is self-explanatory except one part – getting a cached Workspace. Now you might think this is the most simple part but when taking into account that sometimes connections fail, servers are down and Murphy is constantly prowling than you must be ready for that “What If…”. I will write more on this in one of the next posts.

The rest of the code is pretty much self-explanatory and can be found in my CodePlex Project, or directly here.

Resources:

MSDN: How to: Create Custom Configuration Sections Using ConfigurationSection

ArcSDE: Connection to the Geodatabase

2011-12-18T09:46:00.000-08:00

When connection to a database SDE there are two ways to achieve that:

(From ESRI site the ArcCatalog connection screen)

1. Regular connection using Server, Service (enter the SDE service number), database, user(optional), password(optional)

The regular way uses a service on the database machine which must be running. The service can be found in:

Right click on My Computer –> Manage:

Computer Management –> Services And Applications –> Service:

By Default the service will be called “ArcSde Service(esri_sde)”:

As you can see I am not using the regular way (the service is not marked as started). For both 10 and 9.3.1 ESRI advices not to use it this way since it does all it’s work on the DB machine.

Note: if you are using several sde services than the Service port will differ than the default of 5151. If you are unsure about the number than look in the DB server at services file (located at: c:\Windows\System32\drivers\etc and has no extension). The file contains lines for services, their port and connection protocol:

esri_sde 5151/tcp #ArcSDE for SqlServer

2. Direct Connection using Server, Service, database, user(optional), password(optional)

The difference in the connection is only in Service, for example in SQL Server: “sde:sqlserver:<sql_server_instance>” (for any other server use this ESRI reference the forth step).

This connection doesn’t use the SDE service. It connects directly to the DB and does all it’s work on the client. If you are using only this connection type then I advise you to change the “ArcSde Service(esri_sde)” to run Manually instead of Automatic – it just means that at startup the service won’t be turned on.

Managing SDE connection in a developer team

I have created all my SDE connection but now my team mate Bob needs a connection, What do I do?

Well I usually save the connections in one of the team’s network folders with a bat file called install connections. Here is how I do that:

Copy all the connections from %APPDATA%\ESRI\Desktop10.0\ArcCatalog to your network folder.
Create a bat file with:

xCopy *.sde %APPDATA%\ESRI\Desktop10.0\ArcCatalog /I

(The /I is for creating the sub folder when they don’t exist (like when you never opened ArcCatalog…))

Resources:

ESRI: Geodatabase connections in ArcGIS Desktop

ESRI: Preparing database connection files

ESRI: What is a direct connection to a geodatabase in DB2?

Help about xCopy

MSD tools: Changing Connection and Publishing

2011-12-18T06:47:00.001-08:00

Last month I got a new assignment to develop a tool that changes the layers’ connection in a MXD file and publish it in ArcGis Server. It is fairly easy to do it using ArcMap but we have too many environments/servers and whatever we can automate should be automated.

Since then the team went to a Systematics Event (ESRI local provider) where they talked about the new stuff in version 10.1, namely that publishing will only support MSD files (not MXD files) – so the task got changed a bit. For more information on MSD functionality and Performance considerations.

Finding how to change the connection of the layers was easy enough with the IMSDHelper interface:

private const bool UseRelativePath = true;
private const bool ValidateLayerConnection = false;
/// <summary>
/// Changes the SDE connection string for all the layers in the MSD file
/// </summary>
/// <param name="msdFilePath"></param>
/// <param name="connectionString"></param>
/// <param name="resultMsdFile">result file is relative path</param>
public static void ChangeConnection(string msdFilePath, string connectionString, string resultMsdFile)
{
IMSDHelper helper = new MSDHelper();
helper.Open(msdFilePath);
var maps = helper.GetMaps();
for (var i = 0; i < maps.Count; i++)
{
var layers = helper.GetLayers(maps.Element[i]);
for (var j = 0; j < layers.Count; j++)
{
helper.PutWorkspaceConnectionStringInLayer(layers.Element[j], connectionString, ValidateLayerConnection);
}
}
helper.SaveAs(resultMsdFile, UseRelativePath);
}

We simply iterate through all the maps and all the inner layers and change the connection to the given one.

But what about Publishing the MSD automatically? This proved to be a bit more problematic. It was fairly easy to find a Python script using arcPy that publishes MSD files. But less so on ArcObjects code, in the end I found this forum post which led me to this code (just run it through Google Translate since it’s in Chinese). The code looks like this:

public static void PublishToServer(string mapFilePath, string serverName, string serviceName)
{
IGISServerConnection gisServerConnection = new GISServerConnectionClass();
gisServerConnection.Connect(serverName);
var serverObjectAdmin = gisServerConnection.ServerObjectAdmin;
var configuration = (IServerObjectConfiguration2)serverObjectAdmin.CreateConfiguration();
// release the name of the Service, required
configuration.Name = serviceName;
// release the type of service, such as: MapServer, GeocodeServer
configuration.TypeName = ServiceConfigurationTypeName;
configuration.Description = serviceName;
// Service Description
SetDefaultServerConfigurations(configuration, mapFilePath, serverName);
// add service to the Server
serverObjectAdmin.AddConfiguration(configuration);
// start the service
serverObjectAdmin.StartConfiguration(configuration.Name, configuration.TypeName);
}

This is to set the default values of my organization:

private static void SetDefaultServerConfigurations(IServerObjectConfiguration2 configuration, string mapFilePath, string serverName)
{
    var properties = configuration.Properties;
    // Set the path of the file
    properties.SetProperty("FilePath", mapFilePath);
    // image output directory
    properties.SetProperty("outputdir", "c:\\arcgisserver\\arcgisoutput");
    // image output virtual path
    properties.SetProperty("VirtualOutPutDir", "http://" + serverName + "/arcgisoutput");
    // image type supported by
    properties.SetProperty("SupportedImageReturnTypes", "URL");

(there are more settings that can be found here)

I found that it is best to look at existing services, for that I added the capability of getting your service settings in XML format:

public static string GetServiceConfiguration(string serverName, string serviceName)
{
IGISServerConnection gisServerConnection = new GISServerConnectionClass();
gisServerConnection.Connect(serverName);
var serverObjectAdmin = gisServerConnection.ServerObjectAdmin;
var configuration = (IServerObjectConfiguration4)serverObjectAdmin.GetConfiguration(serviceName, ServiceConfigurationTypeName);
return configuration.Serialize();
}

The output of this method looks like this:

<ServerObjectConfiguration>
<Description>PublishDemoIi</Description>
<Properties>
<FilePath>C:\Install\MsdUtils\Demo.msd</FilePath>
<outputdir>c:\arcgisserver\arcgisoutput</outputdir>
<VirtualOutPutDir>http://server/arcgisoutput</VirtualOutPutDir>
<SupportedImageReturnTypes>URL</SupportedImageReturnTypes>
<MaxImageHeight>2048</MaxImageHeight>
<MaxRecordCount>50000</MaxRecordCount>
<MaxBufferCount>100</MaxBufferCount>
<MaxImageWidth>2048</MaxImageWidth>
<IsCached>false</IsCached>
<CacheOnDemand>false</CacheOnDemand>
<IgnoreCache>false</IgnoreCache>
<ClientCachingAllowed>true</ClientCachingAllowed>
<CacheDir>c:\arcgisserver\arcgiscache\GtmPublishDemoIi</CacheDir>
<SOMCacheDir>c:\arcgisserver\arcgiscache</SOMCacheDir>
<UseLocalCacheDir>true</UseLocalCacheDir>
</Properties>
<Extension>
<TypeName>FeatureServer</TypeName>
<Enabled>true</Enabled>
<Properties>
<EnableZDefaults>false</EnableZDefaults>
<ZDefaultValue>0</ZDefaultValue>
</Properties>
<Info>
<WebEnabled>true</WebEnabled>
<SupportsMSD>true</SupportsMSD>
<WebCapabilities>Query</WebCapabilities>
</Info>
</Extension>
<Extension>
<TypeName>...</TypeName>
<Enabled>false</Enabled>
<Properties>
</Properties>
<Info>
</Info>
</Extension>
<Info>
<WebEnabled>true</WebEnabled>
<WebCapabilities>Map,Query,Data</WebCapabilities>
</Info>
<IsPooled>true</IsPooled>
<MinInstances>1</MinInstances>
<MaxInstances>2</MaxInstances>
<InstancesPerContainer>1</InstancesPerContainer>
<WaitTimeout>60</WaitTimeout>
<IdleTimeout>-1</IdleTimeout>
<UsageTimeout>600</UsageTimeout>
<CleanupTimeout>30</CleanupTimeout>
<ServiceKeepAliveInterval>-1</ServiceKeepAliveInterval>
<StartupTimeout>300</StartupTimeout>
<Isolation>high</Isolation>
<StartupType>automatic</StartupType>
</ServerObjectConfiguration>

As you can see the properties in the XML correspond perfectly with the properties settings.

To test the creation of the service you can look at ArcGis Server Manager:

http://ServerName/ArcGIS/manager/default.aspx

Where you can tweak the settings (as you can see I had 5 tries till I got it right…).

Also you can test the service using the REST API :

http://ServerName/ArcGIS/rest/services

Looking into the FeatureServer link you will find in the bottom “Supported Operations: Query” where you can test the Feature service queries:

Some points:

Using PublishToServer needs MSD file located on the server.
In order to use PublishToServer user must be an admin on the server (otherwise the call gisServerConnection.ServerObjectAdmin would fail)
You must clear ArcGis rest API cache before using the service (or you won’t see it in the rest/services url) – for doing that automatically see my previous post: Clearing ArcGIS Server REST API Cache

I have posted the complete tool in my Codeplex project: ArcGisServerTools, MsdUtils, ArcGisServerUtils.

Resources:

ArcGis Server Forum: Promote map service from one server stage to another in automated fashion?

Findleaf - Chinese Blog code

ArcObjects Forum: Publish MXD to server in ArcObjects C#

Keywords: MSD, ArcObjects, ArcGis Server, ESRI

IceRocket Tags: MSD,ArcObjects,ArcGis Server,ESRI

Active Window Loses Focus problem (Full Screen minimizes)

2011-12-11T04:57:00.001-08:00

For the past several years I have been having a problem with my home computer (Windows XP), the problem is not so severe just annoying (though I guess for some people it will be much more severe). My problem? from time to time the application I work on losses it’s focus for a second and then either:

a. gets it back (when it’s in windowed state)

b. another application gets the focus (when it’s full screen applications).

My solution to the problem was using only windowed applications but it is too annoying when watching movies or sometimes impossible when playing games.

From time to time I would search the web for solutions and find forums that advise people to open Task Manager (or other similar applications) and just see what application is stealing the focus – the problem is you need to be fairly fast to catch the bugger (especially when games and movies demand your full attention). Other solutions were with specific applications that caused the problem like AVG anti virus (in one of its old versions). I have tried formatting the computer but the problem returned (meaning it must be some application I always use).

So I have decided to fix (or at least find the problematic application) myself. I first found a windows function that partially does what I wanted – GetForegroundWindow, then searching for it with C# I found a solution at StackOverflow site a bunch of invoke methods that return the current focused window information, all that was left was wrapping it in a console application and using it.

In the end running the little app at home found the problem:

27/11/2011 20:22:10:C:\WINDOWS\System32\rasautou.exe -> Connecting Cables012...

For the past several years my computer has been the dial up computer for the internet. Cables012 is that connection. Thinking to solve connection problems I enabled “Establish a dial-up connection whenever a computer on my network attempts to access the Internet”:

I thought to myself that this option will just reconnect to the internet when the connection is down, well it does that and also steals the windows focus whenever it feels like and at exactly midnight it also activate what I like to call “Blinking Window Mode” when at least one window will appear in the general top left area of the screen and disappear before you can read what it says…

Well the fix was easy enough just uncheck this option and restart the connection. This whole thing was a surprise since I was sure I had some kind of funky spyware app installed which caused all of this…

I have added the source code for this little app to my Codeplex project, and also a release exec (for the non programmers who reach this site).

Keywords: Windows, Full Screen, Focus

IceRocket Tags: Windows,Full Screen,Focus

Clearing ArcGIS Server REST API Cache

2011-11-23T09:29:00.001-08:00

Whenever you change a layer in the DB or publish a new server you might notice that the REST API will not automatically update. In order for it to update you might need to clear the REST API cache.

There are several ways to do so:

The default way:

ArcGis Server comes with a REST API admin page, which can be found here:

http://SERVERNAME/ArcGIS/rest/admin

Logging in to this page gives you the Clear Cache Options:

The page has a bunch of options for clearing the catch semi-automatically:

What you need to do is simply click on the "Clear Cache Now" several times (at some version of the Server one time was not enough…).

Disadvantages:

1. You need to have the admin user/password in order to clear the cache
2. The process needs to be done manually.
3. If you have multiple server you will have to do so for each server

The token way:

ESRI gives another way which is to use a generated token and then you can simply call a link whenever you want to clear the cache. More on that here (ESRI refers to this method as “the easy way”).

Disadvantages:

1. Different token for each server
2. The process needs to be manually set at each server

With Console Application (with code):

The problem with all of these methods is that you can’t really use them to install servers, at my company we have more servers then flies and having to clear the cache after each installation is tedious work that is often forgotten (and when it is forgotten I find myself wasting hours trying to debug the “problem”). So I decided to write a console application that does the clearing of the cache for me. This idea actually came from this forum post, the only problem was that the code didn’t really work for me…

So I tweaked it a bit with WebRequest and the end result is this:

public static string ClearCache(string serverName, string userName, string password)
{
var result = HttpUtils.PostRequest("http://" + serverName + "/ArcGIS/rest/admin/login",
new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("username", userName),
new KeyValuePair<string, string>("password", password),
new KeyValuePair<string, string>("redirect", String.Format("http%3a%2f%2f{0}%2fArcGIS%2frest%2fadmin%2fcache%2fclear", serverName)),
}, String.Format("http://srvp7d-gtm/ArcGIS/rest/admin/login?redirect=http%3a%2f%2f{0}%2fArcGIS%2frest%2fadmin%2fcache%2fclear", serverName));
//The response is in html, only return the response if its not a success
return result.Contains("Cache Cleared.") ? "Cache Cleared." : result;
}

And HttpUtils.PostRequest looks like:

public static string PostRequest(string url, List<KeyValuePair<string, string>> paramters, string referer)
{
var req = WebRequest.Create(url);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
var parameters = GetParameters(paramters);
//We need to count how many bytes we're sending.
var bytes = Encoding.ASCII.GetBytes(parameters);
req.ContentLength = bytes.Length;
((HttpWebRequest)req).CookieContainer = new CookieContainer();
if (!String.IsNullOrEmpty(referer))
((HttpWebRequest)req).Referer = referer;
var os = req.GetRequestStream();
os.Write(bytes, 0, bytes.Length); //Push it out there
os.Close();
var resp = req.GetResponse();
if (resp == null)
return null;
var sr = new StreamReader(resp.GetResponseStream());
return sr.ReadToEnd().Trim();
}

I think that this way is the best since all I need is the server name, the admin user and it’s password. I can do it remotely and I can do it automatically after each deploy. The only thing that is not so good is the using of the clear password (and lets be honest here the next step is putting the call to the console application in a batch file for the development environment and giving that file to the rest of the team so that a hacker…).

Source code can be found at my Codeplex project, here (the code has a few more useful commands like publishing a MSD file, changing the connection of MSD file etc.).

Resources:

Clearing the ArcGIS Services Directory cache “the easy way” – ArcGIS Server Blog

Clear cache through code by demand – ArcGis Server Forum

Hanselman: HTTP POSTs and HTTP GETs with WebClient and C# and Faking a PostBack

Keywords: ESRI,ArcGIS Server,REST API

IceRocket Tags: ESRI,ArcGIS Server,REST API

ArcObjects: Workspace

2011-11-15T04:42:00.000-08:00

This post is the forth post in my ArcObjects series.

As I have written in ArcObjects: Introduction, using ArcObjects is one way of performing CRUD operations against a Spatial DB (with SDE layer on top) and the way to do that is by using the intefaces of WorkspaceClass.

ESRI definition for the Workspace is this:

A Workspace is a container of spatial and non-spatial datasets such as feature classes, raster datasets and tables. It provides methods to instantiate existing datasets and to create new datasets.

Mine is a bit more simple, the WorkspaceClass is the class that does all the “work” within the “space” it is defined to work at. The Space section of the definition can be a DB Sde, File Geodatabase, Cad files and anything else ArcDesktop applications can open. The Work section refers to any basic action that can be done with ArcDesktop applications.

For example: IFeatureWorkspace allows you to create a new Feature Table, open an existing Feature Table and with the help of IFeatureClass perform CRUD operations on that table.

One important note however is that you cannot create a new instance of it, it must be returned for you – the next post will be on getting the workspace.

As you can see the WorkspaceClass implements a lot of interfaces, the good news is you don’t need most of them (I have only found usage for two of them).

What do you need then?

IFeatureWorkspace – probably the most important interface, will allow you to edit features in the Space section
IWorkspaceDomains – allows the creation, deletion of domains (important only if you want to automate the deployment process and using domains)

So how do we use it?

protected internal readonly IFeatureWorkspace _workspace;

protected IFeatureClass GetFeatureClass(string layerName)
{
return _workspace.OpenFeatureClass(layerName);
}

This actually the only thing you can do with just the IFeatureWorkspace, but if you actually want to do something like return all the feature in a layer you need to use the IFeatureClass:

private const bool IsRecyclingCursorInGetFeatures = false;

private IQueryFilter GetQueryFilter(string whereClause)
{
return new QueryFilter {WhereClause = whereClause};
}

protected List<IFeature> GetLayerFeatures(string layerName)
{
var result = new List<IFeature>();
var filter = GetQueryFilter(EmptyWhereClause);
var featureClass = GetFeatureClass(layerName);
using (var comReleaser = new ComReleaser())
{
var cursor = featureClass.Search(filter, IsRecyclingCursorInGetFeatures);
comReleaser.ManageLifetime(cursor);
var feature = cursor.NextFeature();
while (feature != null)
{
result.Add(feature);
feature = cursor.NextFeature();
}
}
return result;
}

So what do we have here?

line 4: creates the IQueryFilter – this goes into the where clause of the SQL we will send to DB.

line 5: we get the IFeatureClass of the layer

line 6 and 9: this is important ArcObjects use Com which is a resource, every resource we use must be disposed. At kine 6 we create the disposing class and in line 9 we attach the cursor to that disposing class, when line 17 arrives the using statement will call the Dispose method for ComReleaser and automatically dispose our cursor. More on cursor and disposing of the resources will be written in a later post.

line 8: creates the cursor that we will use to iterate the rows of the DB, IsRecyclingCursorInGetFeatures defines the way the cursor works (more on that on a later post)

lines 12-16: iterating the data

And that’s it.

IceRocket Tags: ArcObjects,Workspace

Simple Web Mapping

2011-10-10T09:04:00.000-07:00

A friend of mine called me a few months ago and asked for advice on implementing a simple mapping application on the web. He remembered I worked on a mapping project and wanted my help.

His client wanted him to implement a web based site with a map of the client stores. The map should be customizable, meaning you could add items and remove them. And when I write map I mean a map of the interior of the store not it’s location in the world.

Now all the mapping options I know of are bulky and not quite simple but I promised him I will look it up for him.

The first thing I did was Googling “web simple map” got myself this results:

http://gis.stackexchange.com/questions/2800/suggestions-for-simple-mapping-platform-for-alumni-web-site uses Google maps to store the location of people
http://developers.cloudmade.com/projects/web-maps-api/examples

But the options were neither simple nor easy to implement.

Then I got back to the requirements – mapping a store. That is almost like my B.A. final project of internal design for apartments, that had a 3D look to it but the design mode was at it’s base a 2D painter. And that is what the requirements wanted a Painter that has only basic drawing tools like placing certain items on the canvas.

The background for this application will be an image of the store. Saving the canvas will save the items’ locations in the canvas, while loading will repaint them in place.

And the best part there is already an open source Painter application around in Java Script Called Canvas Paint (a copy of Microsoft’s Paint application in the Web) the script is also included here:

The File->Save option allow saving the image on the server…

In the end my friend decided to create a demo using this site, which allows you to place points on an image:

(when the mouse is over a point you get a tooltip with an image)

If the client likes it he will actually program something similar.

So, I guess what I am saying is check out the requirements before you jump to implementing a GIS solution. Mapping for the client doesn’t always mean using world coordinates (and often the client won’t even know what world coordinates mean).

Keywords: mapping, simple, paint

IceRocket Tags: mapping,simple,paint