Sunday, December 15, 2013

From concept to reality in 6 weeks - part 21

Yesterday I took the day off and read a book "The Mountain of Light" by Indu Sundarsan which is a different view on the same story that my favorite author William Dalrymple wrote about called "Return of a King". It was very relaxing to take a whole day to read this 300+ page book.

I definitely did not intend to work today, but when I found out that due to budget constraints I would need to finish this week. Running out of money is one of things common things in life that we just have to deal with. In a perfect world, I could just go on forever writing stuff that the customer wants but someone has to pay for the work being done and in this case, as in all of the cases I have work on,  there is limited amount of funds.

My goal today was to allow multiple server requests to retrieve data for at least 24 hours and have it show up on a map. Seems like a reasonable thing to do since I have to have that working by Wednesday.

First I was curious about a couple of things I have seen in sample code. What is the difference between strings surrounded by double quotes and those with a "@" preceding the double quotes? That was easy to find the answer as it was the first match from Ms. Google as it is called Regular vs. Verbatim strings. The next question is about regular expression string matching. I found a page outside of MSDN that was a helpful start on C# regular expressions. I found good documentation that showed me that it is exactly as I expected. The regular expression quick reference was what I needed.

As I started coding I quickly did something evil again as I kept getting a run-time null exception. Man those are so hard to track down. I went back to see if there was a way to capture every exception and tried both "catch (Exception e)" and an "catch" without any exception. There was something I was missing as one of my asynchronous calls must have gotten me in trouble as I could not track it down. I did learn to that there is an easy way to show the class name in debug statements by using "this.GetType().Name" as that was definitely helpful. One other thing I was a bit concerned about was extending an array as more data became available, but I learned that was no problem as a List can be changed on the fly by either increasing the Capacity or just adding more elements one at a time.

I decided it was time to completely rewrite the complex code I wrote that gets data from a server. The Background Transfer code I wrote was way too complex to debug and was causing too many issues. I went back to the drawing board to learn how to use the HttpClient in a Windows Store app. My code was just so much clearer in the end and actually worked to transfer 24 hours worth of data in 1 hour increments. I was so happy I had to send screenshots to the customer even at the late hour it is.

This post clearly does not cover everything I did as it took me all day long to rewrite my code, so the number of words in this post clearly does not express how much I got done today!

Saturday, December 14, 2013

From concept to reality in 6 weeks - part 20

So here we are at day 20 with only 3 calendar weeks left. Kind of hard to believe that 4 weeks ago today I started on this project. I am definitely over the hump now. The app is actually very stable and makes a good demo just as it is today. But of course I cannot stop today as I need to keep going. I have a couple of things to change this morning related to the latest nightly build from the mapping engine guys. As always the lead mapping engine developer was awake as I started my early day at 5:30am and he wanted to check I had everything before he went to bed. It is a privilege to work with people like him as he does such great quality work. Every day he sends me not only the build but an example app that shows how to use the new features. I will definitely miss working with him daily as it is a breath of fresh air to work with him. It reminds me of the time I worked at SGI as that defunct company was full of the brightest people on the planet. That just proves that because you have transforming technology and the greatest developers anyway, it does not guarantee success. Now every computer and mobile device has an OpenGL supported. Enough of waving eloquently, on to the Windows Store app.

Today was an interesting switch for me. I need to add the latest mapping engine binaries to my project to verify it was working and add some new code to show the selected point on a map. That was pretty easy, but then I got tired of the standard Windows Store MessageDialog as it is a big white bar across my screen. Plus it is modal which means I cannot do anything else while it is up. What I really want is a custom popup dialog that I can tap all over the map and update the contents of the popup as I tap around. If the user close the popup and then taps on the map then I want to re-show the popup dialog. How hard can that be? Apparently I had a good reason is using the MessageDialog as it was the easiest thing to do. Now I get to use this as a test to learn about notifications and bindings.

I need to switch to some more subtle music to get thru this next section so I start listening to Robben Ford & The Blue Line on Spotify.

The first thing I had to figure out was how to style the popup. I want a white background with a blue Close button in the bottom right hand corner. That should be easy. Turns out it is easy to get the layout as I can use the Visual C# Blend editor to help me with the design but then for reusable styles I have to create static resources in a XAML file which means editing XML. I guess I have too many years of doing HTML and CSS as I expect this to be easy. On the web I set a background color, a foreground or text color and then I change the colors for pseudo elements like hover states. Easy as it should be. The first thing I try in the Blend editor is to code complete on the properties to see if I can get the hover styles to work in Blend. It turns out that is a bad NO. I can easily set the background and foreground colors to make it close. I want to explore the existing styles that are available in Windows Store apps as I like the way the MessageDialog text is styled by default. Wait, I wonder if I can just style the existing MessageDialog? It turns out that is the big NO also. There are MSDN social blogs available where I keep looking but it just is not possible. I keep hoping to find something but the more I look the more frustrated I am becoming, so I just give up and realize I must create a custom dialog. Microsoft does not even publish the styles so without knowing the keys to the Microsoft kingdom you cannot enter that gate. I start to look into styles within Blend as I have seen code like this before:
Style="{ThemeResource BodyTextBlockStyle}"
So what do these styles actually look like? Do I have to set them and then run the app to see what it looks like? That is pretty sad, so I go looking for my answer on line and find this interesting site that shows some of the undocumented Metro styles. That helps a little bit but let me see if I can find button styles as I need to find the hover like styles in Metro UI and I find this social blog answer to my stylish problems. Now that is just about as complicated and unobvious as I could imagine. The only good news is that it actually works. The real answer is that you need to start with the existing default styles and modify them as you have to have the correct XML hierarchy or else nothing will work. I did find an article that described how to do this programmatically but I am just not going to go there. This is way more complicated that I need so why would I go all out and really torture myself? Just to be clear Microsoft does document the styles so for a button everything you need to know can be found if you know what to look for as they are called themed resources. This is an example of the complete documentation for TextBox styles and templates. I experiment with transparency which in Windows Store apps is an Opacity property to see if I should make the background for the dialog slight opaque and the answer is no as it does not look good. So I move on. All in all I found success on styling my custom popup and I did learn how to do it correctly but I never expected to wake up this morning and have to go thru all of that mess. Now onto notifications...

I need to connect my model classes to change the popup dialog contents and to be notified when it closes to do some cleanup. I am expecting this to be complicated. I like the notification manager in iOS but I am no way expect the same in Windows Store apps. The first thing I need to get a handle on is bindings. I have also see them in many examples over the last few weeks, so I need to make sure I understand how they work. I found yet another quick start guide on how bindings work that was useful. I then found the explanation of property notifications which all made sense. So you have a model that sends notifications when a property changes and the view is observing the model and receives the notifications. That all makes sense. Let me try to explore creating custom controls and then hook all of that up using notifications. I found an example on how to create a custom control which was helpful. I move all of the custom popup code to a new user control class. I then try to use my custom dialog in my main page and I get the strangest error:
Inconsistent accessibility: return type '...' is less accessible than method
What on earth does that mean. I go to the internet for help as it sort of makes sense but I don't know how to solve it. I found an explanation on stack overflow but it just feels odd to make my control public so I can use it in my main class. I quickly contact a co-worker to help me thru this but after a couple more hours of frustration on my lack of understanding, I give in a revert to working without property notifications as I understand the principle but in practice I am not man enough to get it working. I think I am just overly tired and under too much time pressure to think clearly. The last trick to get the easy way to work is that I have to learn how to pass a function as an argument to a method as my callback on when my dialog closes and I found a good explanation on my friend Mr. StackOverflow.

I just need to call it a day as this was my longest day so far and the long days are going to have to end soon as I am getting run down physically and my whole family is falling sick so I don't want to succumb to that sickness mess.

Thursday, December 12, 2013

From concept to reality in 6 weeks - part 19

Well today is a new day. The days of coding are getting longer and the amount of daylight is getting shorter with each passing day as the day with the least amount of daylight is coming soon. It is so cold out to me that it is not very appealing to be outside anyway!

Today I have a new mapping engine that I need to add to my project and then see how many things have been fixed. We were having issues with touch events working but today that has been fixed. That is a great start to a new day. I also added a debug statement handler for the mapping engine so when running in debug I could tell if I caused any errors by incorrect setup on my part. The mapping engine team of two needed that functionality anyway to help debug crashes I was seeing so hopefully it will help us both.

I had yet another victory using LINQ and the more I use it the more I like it. This time I needed an "orderby" and "group" to find unique values. Oh my gosh that was so simple and makes sense at the same time. Now if I just had that in other languages that I use. This list of LINQ grouping examples definitely helped me a lot. Then I wanted to pass the group to a method to do more calculations and I found that I can create a List instead of having to deal with IGrouping by using ToList(). Wow that is so nice.

Next up is a little bit of math and thanks to the iPad app code I learned about this great site that explains in exact details how to determine a bearing. That certainly was easy, except for one thing I need to get the floating point modulus, which does not exist in the Windows Store Math library. Wait, it turns out that the "%" operator already handles that and so there is no need for a special Math library function like in Objective-C - that is the way it should work. I think I am finally getting the hang of this C# stuff. Of course it matters who you hang out with. Spending many hours yesterday with ex-Microsofties helps me know which language is better!

Pretty happy day so far as I finished the last layer on the map so now just need to look into a couple of small bugs and move onto the next tasks of filtering the data being displayed on the map. That is after I get the app working on another Windows 8.1 machine to make sure all of the mapping engine bugs have been fixed that preventing me from getting that far yesterday.

Today I am trying new music by following related to links in Spotify and found Doyle Bramhall again and I am enjoying like while struggling with builds. And then something magical happens as I have the app running on the other tablet. Maybe I should just quit while I am way ahead and it is not even noon yet.

I took a short break to celebrate but helping my daughter with my old Mac laptop as it was acting strangely. It is a 2009 model and still works pretty well for homework and cruising on the internet so it has done a great deal of work over the years. I originally bought it to work on writing my book as I wanted to use Pages and carry it around with me at work and when I went to the library. Overall is looks pretty good and recently my daughter decorated it with a turquoise outer shell and has a thin rubber key cover as well. All it took was a reboot and then holding down Command+S and I checked the hard drive to make sure there were no problems. All was well in the old Mac land.

Now I am back to work as I need to get filtering working. Wait, I cannot start yet as my Windows laptop has a bunch of updates that require a restart. I might as well do that before I get back into the depths of filtering. Just when I thought the work would begin on filtering I re-ran the app and noticed a point off the west coast of Africa, which can only mean one thing. There was a slight problem in that I always need to check to make sure every GPS point I receive is valid. Not that a point off the west Africa is not valid, it just is not valid in my case. I had to do some searching to see if there was anything special I had to do in C# to test a float is close to zero. Turns out the solution would work in pretty much any language.

Had to take a break and do a show-n-tell at work so others could see all of the amazing stuff that is working in the 18+ days so far.

Now I get to learn about notifications, bindings and the event model used in Windows Store apps. There is a couple of articles I need to read to become familiar with it.

At the moment I have to figure out how to take a screenshot on a Windows tablet without a keyboard. This is special hardware I am on so hopefully a generic solution will work. I searched Ms. Google for "windows 8 tablet screenshot" and found an article that described exactly what I need. Thankfully clicking on the Windows button on the front of the tablet and the Volume down button did the trick. It is kind of like the Home button and the Power button on my iPad. The only problem is I have to look to find which button is the Volume down button as the up and down buttons feel just the same. The Power button is right above the Volume up so that is probably not a good button to click on randomly. There is another button below the Power button that I don't even know what it is. I click it and behold it is the Autorotate button, which feels different from the others as it is smaller. The small unreadable label did not help me figure out that one.

I think it is time to call it a day and think about the notifications tomorrow morning when I am fresh from a good night's sleep. I am a morning person and need such tasks that require concentration for the wee morning hours.

From concept to reality in 6 weeks - part 18

In two more days it will be one more week gone and that will leave me with only 3 weeks to go. No pressure! After a very long day yesterday, I definitely needed a night to rest and just not think about the Windows Store app. I woke up this morning ready for a new day. It was super frosty outside and I was just glad I did not have to go outside. Definitely zero interesting in running when it is this cold out.

The first thing is to solve the problems I had with one of the layers not displaying the correct colors. I had to go into the code that was defining the values and put in a "System.Diagnostics.Debug.WriteLine" to see what the values were. After seeing the output it was obvious what happened as I had two problems. The first was I saw a bunch of "-Infinity" values. I handled the case with missing values in my data loader but forgot I need to ignore those values when displaying data. The second problem was that all of the values were fractional and yet I stored color values in the range from 1-10. A quick scale factor solved that problem.

Then a couple more fixes and one last test proved that something had gone seriously wrong. I am now getting a run-time exception as the mapping engine does not like something I am passing to it. I try to use the Git client to see what I changed that was not good. I did not find it so it is time to debug until I find the problem.

I had a GoToMeeting with the customer, who was a no show, but I needed to document some of the potential UI needed for the next piece, so I added screenshots of the iPad app into Kanbanery along with some text explaining what I need to do. One of my favorite characters is the checkmark which is needed for the text description of the next few tasks, where I needed to show checkboxes [].

Now I have to hurry up and find this run-time error somewhere in my code. First I need to check on updating the UI from a background thread since I want to make sure that is not my problem. Microsoft has some nice documentation on asynchronous programming that explained many useful things. I don't see anything I am doing wrong so time to dig in a debug until I find and fix the problem. Some times you just have to relax, take a deep breath and then dive in.

I have to say that finding run time exceptions is just nasty in any language. I finally found and fixed the problem as I was calling an asynchronous method and when it returned I had assumed it fully completed - what a terribly bad idea that was. I looked back at the Git source history to see all of the changes I made this morning and then noticed the "async" change. I got into the debugger and checked the values after the method returned and found nothing was setup correctly. Bingo, found that nasty bug and now I get to move onto the making another build to test on another machine that mine.

Since I have only create one successful release build, I had to go remind myself how to do that by reading the documentation. The big secret is to select Store > Create App Packages... and then select "No" that I don't want to put the app in the store. My executable is now 10MB in size and that is after I removed BingMaps which made no difference at all in the size of the executable. Hopefully someone other than me will be able to get that to work correctly in Windows 8.

Turns out that it worked sort of. I was able to load the executable on a USB thumb drive and install it on another Windows 8.1 tablet but the way VSE creates a package installer is a bit odd. You have to be the administrator of the computer in order to use PowerShell. I found a Microsoft article that described the ticket to PowerShell working as you have to set the execution policy as in a PowerShell window:
Set-ExecutionPolicy RemoteSigned
This allows the command to be run to install the executable in the Metro desktop by running:
./Add-AppDevPackage.ps1
Of course most fonts make it very hard to know whether that is a numeric one or lowercase "L" as the last character in the file extension, but that is another usability problem for another day.

Most of the remaining part of the day is trying to get work done in a very crowded Starbucks. I was meeting the mapping engine developers there. Typically they told me it was empty, but when I arrived there was not a single seat available at 2pm. I was constantly getting a dropped WiFi connection, which I assume was due to so many people trying to get WiFi and since I did not use it constantly I was being dropped. I finally gave up and just used the WiFi from my iPhone as it was the only reliable WiFi. Then it started having issues.

It was time to figure out how to get the app working by reading the files I downloaded from the server as that was my intent all along but did not take the time to get that working. For some reason the super crowds in the Starbucks must have caused my brain to stop working as I had to read about File IO again. The quick start guide to reading and writing files in a Windows Store app to the rescue. Everything hinges on a StorageFolder and a StorageFile. Finally after an hour I had the JSON being read from the last files I downloaded that worked. Not bad but it should have been a couple of minutes since I had already dealt with it, but so much time and code had passed thru my mind since then I guess.

The rest of my day was learning all of the new mapping engine APIs that I needed to call and making notes in Kanbanery so I did not forget how to do them when tomorrow morning arrives early. I also had to deploy many times and try to get the app running on a couple of other Windows 8.1 machines to verify the whole deployment would work easily.

Tuesday, December 10, 2013

From concept to reality in 6 weeks - part 17

Today we start yet another day but for me it is all down hill from here on out. Today's big challenge is to make another build and see if my customer can install it without it prompting for my developer account. I also need to explorer ways of installing a Windows Store app outside the store to make sure the Microsoft documented way is the only way to get it to work. I also need to get layer filtering to work with the latest mapping engine build. On this very raining cold day I am expected to get a lot done. In general I need to hurry up as I have a week until the next full moon day and we all know what happens on those days. Before jumping into it, I have been re-listening to the music of Roy Buchanan while working this week. Yet another unknown talented guitar player who met his early demise. It just seems like the most creating people always have the biggest problems and Roy Buchanan killed himself in jail many years ago after yet another drunken state. So sad for me to read this again.

I started the morning by double checking all of the anchor points for the images I created yesterday to make sure it was exactly the same for each one. That was a bit time consuming. I then had to create white versions of each icon as that is what the mapping engine requires to color the values by height. It was time again to learn a bit about ImageMagick and how to do this in an automated fashion. This required using a trick where you pass an opaque color and the fill color. I preferred to ignore all pixels except the transparent color to do the fill that way:
convert +opaque transparent -fill 'rgba(255,255,255,255)' A.png A_white.png
I learned this trick by going to one of the great tutorials that explains how to use ImageMagick for this purpose called Color Basics.

I wish there was more to write about today but it was just a tons of coding and debugging. Nothing new or exciting to write about. All in all a good day as I got most of what I wanted done. I did not get time to look into how to deploy the app outside the Windows Store as that will have to wait until tomorrow.

Monday, December 9, 2013

From concept to reality in 6 weeks - part 16

Today is thankfully a new day as I just want to completely forget everything that happened this weekend as it was all just a bad dream, or at least that is what I want to tell myself. It was all personal problems with my family that is just getting worse with every passing day. That is the challenge of balancing family and work life. No client of mine pays me to moan and groan about my personal and family issues. They are paying me for super first class high quality results. Part of experience of getting old is to be able to separate family and work and put 100% effort into work every when things are going terribly at home. I really learned this two jobs ago when my best friend was dying of cancer in front of me daily at work. He definitely rose above his circumstances and did his absolute best even when feeling horrible from chemotherapy or radiation. He was a trooper until the end and I have to remember those 3 years for the rest of my life.

Now it is time to refocus on the task at hand as I am half way thru the project. Elapsed time wise I have 4 calendar weeks left to go. I realize that does not mean I have 20 days as I want to spend time with my family during Christmas and New Year's Day. Today my task is to finish off the final layer and resolve the layer icon issues so they all look correct by color and size. Before I start on that I need to check the label changes I made and learn about latitude for the millionth time it seems. I quickly go to Wikipedia so I can see a visual picture in my head so maybe just maybe I will remember this time that longitudes or meridians go thru the poles and latitudes are parallel to the equator. I change my debug statements to show the extend of the markers for N/S latitudes and E/W longitudes. I then need to fix my labels to show altitudes as whole numbers as I see decimal points being displayed, so I switch back over to the C# number formatting documentation asI have been there before. The secret was to use "N0" to show a number with no values after the decimal point. Two problems resolved just like that.

Now to solve the markers. I have decided to automate this whole process by using ImageMagick convert program since I don't want to edit the SVG files and generate PNGs each manually as that is too time consuming. I had to regenerate the triangle in Inkscape so I had to look that up as it is not obvious how to do that. The first thing is how to set the transparent pixel which is done with -transparent option. The second thing is how to account for the case when the background color is not an exact single color and that is fixed by using the -fuzz option. Then I need a way to change the default foreground pixel color as it needs to change for each marker depending on elevation and that is done with the -fill option. Now I just need to figure out the colors that match each of the icons and I am done. ImageMagick definitely helps me here as it supports many color syntax values. I have RGBA values stored in C# so I am going to use those values in my Bash script that will auto-image-magickally do every thing for me. As an aside, this has to be one of the worst names in all of UNIX, the shebang marker at the top of the UNIX shell script which I add to my script. I always run it with the time command so I know how long it takes to run:
$ time ./convert.sh 

real 0m1.388s
user 0m0.443s
sys 0m0.278s
This is what my script looks like in general:
convert -fill 'rgba(181,13,33,235)' -fuzz 20% -transparent white A.svg A.png
The last thing I had to do in my script was copy each output PNG file into the Windows Store project Assets folder so I can easily add them into the VSE project on my Windows laptop. I am still not happy with the icons as I need to find a way to use vector icons as generating the PNG files just don't look good at all. The functionality is working, but it looks bad, so I need to address that somehow.

The whole rest of the day was fixing, changing, editing and resizing icons to make sure they look really good as they are essential to the app. Gimp and Inkscape were my constant companions all day long. It took way too long to fix all of the images it had to be done either programatically or manually. I am just glad that it all looks good and I can call it a day.

Saturday, December 7, 2013

From concept to reality in 6 weeks - part 15+

With my customer visit yesterday I did not have time to record my thoughts all day long. This means I get to try to remember every thing that happened yesterday and this morning. I woke up early this morning so that should be pretty easy to recall.

The first task I completely failed on yesterday was switching from Kanbanery to Trello for the Kanban project management board. It is just so hard to switch tools like that in the middle of a project. I would need to export all of the tasks from Kanbanery and import then into Trello. That is just not going to happen easily as I have almost 50 tasks in 4 different columns. Something is going to get lost and I don't have time to fix stuff like that.

Part of this story is going to be easy as I can look at the Git log and see what I did yesterday! The big news for yesterday is that I got a new build for the mapping engine and was able to get it all working and showing 3 out of the 4 layers. I definitely like it when things just work. I tried a couple of times to get the images working and then just gave up and colorized the ones I needed to a fixed color. I looked at the iPad app Objective-C code to find the colors and the layer settings since we are using the same mapping engine. That exercise just reminded me how verbose and challenging Objective-C is to write an enterprise application. The code just hurts my head and is painful to read and to follow the flow of what is going on. On the other hand C# is very much like Java or ActionScript where you need to understand the framework, threads and events and you are done.

A couple of new challenges occurred yesterday. The first one was I had to learn how to split strings but that was very easy by reading the documentation. Then I needed to figure out how to create a new LINQ query with multiple where clauses. Back to the documentation and within minutes I yet again fell in love with LINQ. As I was adding this new LINQ statement, I was chatting with the mapping engine developer and he told me he worked on LINQ while at Microsoft. I quickly told him, he was now in the circle of trust as my favorite Microsoft developer. I have only met a handful of Microsofties but I have never lived in that part of the country either. Doesn't matter as still anyone involved with LINQ is near the top of useful people.

Another lesson while changing to use the new mapping engine API, was to learn how to read a file from the file system using Windows.ApplicationModel.Package.Current.InstalledLocation. That is interesting since working on web sites for so long you just forget that you can actually read and write files!

Everything considered, yesterday was a huge success so it was a very happy day. I was able to take a break and play tennis also since it was a nice spring day in the middle of December.

The hardest thing all day this morning was fixing images. I still do not have them working as I would like but I quickly made PNG files so I could verify my layers were displaying properly for the data values. I find that the easiest thing to do is use Gimp or Inkscape on my Mac, check the files in and then go to Windows and use Git to retrieve the files and then add them to my project. So much easier than waiting for SkyDrive which is not the speediest tool in the shed.