Podcasts – Mine and Others

Podcasts – Mine and Others

Let me be honest – I hadn’t listened to podcasts in a while, so was pleasantly surprised when I was recording a podcast with Mario Cardinal and Guy Barrette on CosmosDB and, during the initial chitchat, Mario talks about a great podcast he started listening to called After On.

Here is a link to the podcast that I have recorded, it is an introduction into CosmosDB, a service within Microsoft Azure that is a NOSQL and built from the ground up to scale with the web. It is the kind of technology that could be used to build things like Facebook or Twitter.

As for After-On, this is a podcast I instantly became hooked too. The discussions are out of this world, on diverse subjects and very in depth. Please go listen and let me know which is your favorite episode.


Shrinking USB keys to save a old Windows Tablet

At the first Microsoft Build event, they gave attendees a Samsung tablet, made for Windows 8. I loved that tablet (other than the fact that it was heavy) because it had a built-in cellular modem.

But sadly, last week, the thing crashed and was going into a boot loop. The only way to revive it was to reinstall everything…

This was going to be a challenge because this particular model required a 4gb Fat32 USB key. Unfortunatly it is 2018 and the smallest key i found was 16 gigabytes…

So search the internet and i didn’t find an exact recipe, but i did try something that could work – so i gave it a try. Open DiskPart and issue the commands:

list disk
select disk 1 (in my case it was 1, it might be different for you !)
clean
create partition primary size=4160
active

That’s it, then i formatted it, copied the contents of this ISO of Windows 10 onto it and bingo, it booted and in thirty minutes everything was up and running.

To get back the key to 16gb, just run the same instructions without the size-4160 part.

And that’s how you shrink a usb key.


Azure AppService to FTP (not in Azure)

Oy,  I just spent a crazy week to learn that :

It is impossible for an AppService application to connect to a FTP server on the internet (in passive mode). The reason is that each Azure App-Service is assigned a pool of IP addresses for outgoing traffic and Azure is free to choose a new outgoing IP for each connection (the natting stuff) and FTP expects the data connection to be from the same ip as the control connection.

This said, when using sftp, the handshake is negociated with the connection that composes the control channel. When the second connection to transfer data is built, the ftp server receives the connection from a potentially different IP address, which is a 425 type error.

Took us 3 days to diagnose, one evening to write a basic http web api that allows to post and get files, a few hours to install it all and a full day to rewrite the code that worked with ftp…

This said, it was something that no one on the team could of had predicted, live and learn !


Screen scrapping

One thing I believe in is constant change, and constant learning, ideally one thing per day. Sometimes it’s learning how to cook the best eggs benedict ever for breakfast, or sometimes it’s helping out a friend with a special request.

Today I was asked by a colleague if I could help extract data from a web site. As an architect, the first thing I look at in the “code” is a clean separation of what is presentation and what is data. Obviously, I did not find that, which made me realise how frameworks which render html mixed with data are bad bad bad. Why can’t everything follow MVVM with some binding of some sort.

Anyways, we needed a solution and what we wipped up was screen scrapping.

My first attempt was to write a small html page that loads jquery, does an ajax call to hit the webpage we needed data from and then extract it from it’s DOM… Turns out it was an easy to execute but I was met an error : CORS headers not found for file://dlg/extractData.html. GRRRRR

New strategy !

I opened Chrome, did a search for screen scrapping extensions and behold I found this.

An extension that would allow me to navigate to any page, lookup how it is built, understand it’s usage of CSS selectors and voilà. Any page that reuses the same css selector to represent repeating data (as in a list) can be extracted to a json or csv file.

Well, thanks DL for getting me to learn something new today !


ASPNET Core websites, developping with SSL

While working on the backend for To-Do.Studio, we ran into a scenario where we needed to test on a developer’s machine with https (ssl) enabled.

note – we use Kestral for testing and not IISExpress.

First thing we did was to try and add a https url in the LaunchSettings.json file. That didn’t work 🙂

What we found was that had to configure Kestral manually and tell it which certificate to use. The code in our Program class looks like this :

public static IWebHost BuildWebHost(string[] args) =>
   WebHost.CreateDefaultBuilder(args)
     .UseApplicationInsights()
     .UseAzureAppServices()
#if DEBUG
     .UseKestrel(t => {
        t.Listen(IPAddress.Loopback, 55172);
        t.Listen(IPAddress.Loopback, 55000, o =>
        {
            o.UseHttps("localhostCertificate.pfx", "password");
        });
     })
#endif
    .UseStartup<Startup>()
    .Build();

But how do we get a certificate ? There are various ways but i didn’t feel like finding my win32 sdk as some instruction’s on the web… so i decided to use my Ubuntu WSL…

Two commands

erik@ErikLAPTOP:~$ openssl req -x509 -days 10000 -newkey rsa:2048 -keyout cert.pem -out cert.pem
erik@ErikLAPTOP:~$openssl pkcs12 -export -in cert.pem -inkey cert.pem -out cert.pfx

and i had a good looking self made certificate. The hardest part was to copy this cert.pfx file to a Windows directory so i could use it my code.

Voilà ! after modifying my LaunchSettings.json, i could test in either http or https mode !

"ToDoStudio.Server_http": {
 "commandName": "Project",
 "launchBrowser": true,
 "environmentVariables": {
 "ASPNETCORE_ENVIRONMENT": "Development"
 },
 "applicationUrl": "http://localhost:55172/"
 },
 "ToDoStudio.Server_https": {
 "commandName": "Project",
 "launchBrowser": true,
 "environmentVariables": {
 "ASPNETCORE_ENVIRONMENT": "Development"
 },
 "applicationUrl": "https://localhost:55000/"
 }

Windows 10 Redstone 4

Well, as you all know, windows 10 Fall Creators Update is out (internally named Redstone3) and those of you who are Windows Insiders have started getting bits Redstone 4, which should go out in the spring and doesn’t have a public name yet…

In case you are reading Microsoft, this is what i would like to see…

  1. With Windows 10 ARM devices coming out soon…
    • Make Cortana better, make her take more real estate on the screen and not just a small popup where i am always scrolling. Innovate in tablet scenarios.
    • Make Windows Explorer a bit better in tablet mode, maybe tabs
    • Make the tablet experience one or two steps better, i miss swiping from the left quickly to change apps from my Windows 8 RT tablet.
  2. Upgrade Skype
    • Make Skype UWP better, allow to login with Microsoft work account and offer Skype4Business functionality. I don’t want to have two apps doing the same thing. OneDrive managed to do it, you do it also. It would be ok if Skype4Business functionality was in both Skype and Teams.
    • Make SMS sync work for each platform without bugs with mms support.
  3. Finalize Andromeda and Continuum, make it awesome, any Windows 10 device displays itself in the proper way according to how it’s being used.
  4. Upgrade networking, i use a LTE dongle which i cannot even mark as a metered network, innovate in the space somehow to bring innovation in the way in which we connect to wifi networks, lte or share the internet. At the same time, please have an automatic VPN mode for public networks.
  5. Innovate in sharing and continuing work…
    • Cloud Clipboard (was suppose to be in RS3)
    • Windows TimeLine (was suppose to be in RS3)
    • OneNote and Edge on every platform where everything syncs magically
  6. Surprise everyone with Edge
    • make it support PWA and all modern features, you are almost there
    • make it a rock hard stable, you are almost there
    • offer a few more innovations, you can do it !
  7. OneNote UWP should become a premier tool, deprecate the win32 version and make it ubiquitous
    1. Writing on webpages in Edge should not be a Edge feature but a feature of OneNote. It should work in any context over any app
    2. The Ink Workspace should revolve around OneNote
    3. Make OneNote accept personal and work accounts for synching like OneDrive
    4. Get rid of sticky notes,  OneNote should be my sticky notes app
  8. Make up your mind with the UWP office apps (Excel, Word, outlook, powerpoint). Stop having two flavours and be bold. I would prefer the UWP versions because sometimes less is more.
  9. Have a family mode which works on the lock screen or in a guest mode – i see this as a part of Andromeda and Continuum when the device is locked or we don’t’ know who is using the device, allow to browse the web in a inprivate session. There might be something called family apps in the store that are specialised to run without a user being logged in.
  10. Have a dongle based on windows 10 arm that has a hdmi plug, wifi, Bluetooth and a few usb ports. It can be plugged into a monitor to offer a full desktop or miracast receiver experience. Plugged into a tv for a tv experience. In enterprise this could replace thin clients or chromecasts in boardrooms. It also has to be cheaply priced.
  11. Get your media story together, after dropping Groove subscriptions, i feel tv is the next thing. Partner instead of dropping, you need presence in this space.
  12. Office personal and Ofice365 should have deeper integrations into the OS. I feel i am always on the web for these services
  13. Upgrade WSL to have a built-in wayland display server
  14. Upgrade WSL to bring Android apps to Windows 10 all factors, if you can’t beat them join them. Apps should be downloaded from the Store. If there were three major android stores (Google, Amazon and Microsoft), i think we could start moving towards shared api and not the closed Google Play apis… Xamarin could maybe provide that api…
  15. Keep on fighting and have presence in the mobile and tablet form factor, continue the Window S concept for low price points (but always upgradable if necessary)

The art of Debugging

I have been debugging software for the last 20 years, and although things have changed, I realize I took it for granted and never really thought about it.

Basics

There were two debugging technique if was introduced to when I started.
  1. Attach a debugger to a program and hit breakpoints. When you hit a breakpoint, the debugger can show you your source code because the symbols have a mapping from the assembly code to the source code. At a breakpoint you can inspect variables, change their values. You can control the flow of execution of your program by executing the next instructions and even change what the next instruction is.
  2. This technique is a bit more brute force, but consists of using logging to get insight into code which is hard to debug with breakpoints. Although less efficient than breakpoints, I have done this soooo often because it allows me to debug also in cases like production, where I might not be able use breakpoints.

Step forward a few years and we now have Visual Studio 2017. Both techniques above still work but with a few new features. For example, for logging we don’t have to use the console but have access to Trace and Debug, both with listeners to output where you want. Even better are TraceSources, which are specialized objects for logging for modules. Always making things easier, VS also shows us a few graph about memory and threads.

To complement the advances in the platform, there are a bunch of other logging frameworks such as nlog, log4net, serilog… One of the advance i love about these frameworks are the concept of structured logging, imagine logging with DTOs. While we are on the subject, there are even logging frameworks as a service, all cloud based !
There are dump files that can be created at breakpoints in your code, which are basically a memory snapshot of your process. These can be inspected with WinDBG or Visual Studio
It does feel much more modern but all that we have seen is fundamentally the same thing. Things are getting much better…

Modern Debugging

One of the bigger breakthroughs in debugging we got a few years ago was called edit and continue. You can literally change code of running programs as they run… Quite interesting although I cant say have used this mechanism often. This requires Visual Studio.
The second one which has served me well is IntelliTrace. This is basically a configuration based approach to automatic logging – as the debugger hits different places (breakpoints) in the code, events are generated with some contextual information. For example – File.Delete() will generate an IntelliTrace event with the text “File asdf.txt deleted”. This is great as all major components of dot net are instrumented with this technology. IntelliTrace requires Visual Studio but using the offline IntelliTraceCollector, you can record the execution of your code and analysis everything in Visual Studio.
The newest feature which is currently in preview is called Time Travel Debugging, which basically allows your to execute your code… in reverse. You can go forward into your code’s execution as well as backwards, just look at the ribbon in WinDBG Preview (available in the Windows Store).

I haven’t had the chance to play with this too much, but it looks very fun, as it will allow you to go back in time in a break point within a loop… The recorded traces have to be viewed with WinDBG.

Another new feature which i think i will love is the concept of snappoints and logpoints. Imagine the ability to attach to a running process, but instead of pausing the process when you hit a breakpoint, a myriad of information gets recorded when a snappoint is passed. You could then inspect the variables and stack trace of every one of those hits without causing impact. Logpoints are dynamic logging, pass this instruction and add this event to the log. These are great advancements to the art of debugging. This feature requires Visual Studio and Azure hosted sites.

I hope you are excited about the future of debugging !


The Microsoft Graph

One of the project i was working on involves taking Office 365 to the next level. Take the tools that Microsoft gives us and bring it to the next level.

Some of the things we do involves extending Sharepoint, Excel, Word, Office with addons. Works great. But where do we get all the nice information we show the user ?

This is where the Microsoft Graph comes in (http://graph.microsoft.com). It is basically an odata feed that gives you access to a variety of data that represents you. Naturally, you need to send a bearer token to get access so authenticate first !

You can find then entire “metadata” on the internet itself, all self-describing ! Also, there is some documentation.

You can even try the Graph Explorer, which is a web tool to explore the graph in an interactive way.

Here a are a few example of me for my own company :

Who am i ?

GET https://graph.microsoft.com/v1.0/me
{
 "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users/$entity",
 "id": "",
 "businessPhones": [],
 "displayName": "Erik Renaud",
 "givenName": "Erik",
 "jobTitle": "Principal Architect",
 "mail": "erik.renaud@modelon.net",
 "mobilePhone": "+1 514...",
 "officeLocation": "+1.514...",
 "preferredLanguage": "en-CA",
 "surname": "Renaud",
 "userPrincipalName": "erik.renaud@modelon.net"
}

What about my picture ?

GET https://graph.microsoft.com/v1.0/me/photo/$value

 

This is a goldmine of information and makes any business tool soo much more powerful.

Add  an email, find recently used files, show how many unread things in teams… It’s all possible !

 

 


Mocking web services

For one of the projects i manage i have two teams and in the end, one implements web services for the other to consume. In production and integration testing, things tend to go well. But when we are faced with debugging or testing the side that consumes web services, we need something more.

I love do discover new things or new ways of doing things, enter EasyMock (https://github.com/CyberAgent/node-easymock). It is a small nodejs web server that returns any file on disk with extra options in it’s config.json file.

You install it with :

$ npm install -g easymock
And you start it within your work directory with :
$ easymock
Server running on http://localhost:3000
Listening on port 3000 and 3001
Documentation at: http://localhost:3000/_documentation/
Logs at: http://localhost:3000/_logs/

If you wanted to mock a CurrentUserCount rest web service which is located at /api/CurrentUserCount, all you need to do is create a “api” directory with a file named “CurrentUserCount_get.json” within it. Here is that result :

There is even a handy automatically created documentation page:

Happy mocking !