Microsoft .net Orleans

Microsoft .net Orleans

The Microsoft Orleans project (http://dotnet.github.io/orleans/) is a .NET framework for building systems on an Actor Model paradigm.

A typical transactional system receives a command and executes it. Executing it usually means fetching data from a database, modifying it and then saving it.

The reason I was interested in Orleans for To-Do.Studio was that each action in the app generates a command, there is no save button, no transactional boundary. This naturally creates a chatty interface, where many small commands are sent to the server, which much for each: get data, modify it and save it. Combine that with the fact that NOSQL components such as CosmosDB make you pay for reads and what you have is an expensive bottleneck.

The Actor Model in Orleans would have fixed this for me by doing the following. The first time a command executes which references a particular domain object, a grain is activated and the data is read. Each command then modifies the data in the grain. Eventually, when no one uses the grain the system will deactivate the grain causing it to save itself.

This would have the added benefit for us to minimize data access costs and offer speedups similar to adding a caching layer.

As with all magic though, we lose some control:

  1. First thing is that as the system scales to thousands of users and tens of thousands of grains, we have to think of system scalability (Orleans doesn`t yet deactivate resources based on memory pressure, only an “unused delay”)
  2. The deployment model is not fully integrated with Azure.
    1. Hosting is possible within a VM – not PAAS enough for me
    2. Hosting is possible within worker roles – sounds interesting but not exactly what I want
    3. Hosting is possible within Service Fabric (which is another implementation of an Actor Model from the Azure team and not the .NET team) – doesn’t feel seamless this would be my ideal hosting option)
    4. Host in a Linux container which is scaled automatically by Kubernetes – to be honest, I am not a container fan; Ido see their advantages for certain workloads, but it feels like the PAAS of infrastructure people.

Anyways, my best option would be hosting on top of Service Fabric. It would need to be 100% integrated with dotnetcore stuff though.

I could also recommend having a dedicated PASS offering for Orleans (kind of like what happened with SignalR).

Finally, Orleans should be upgraded to support memory pressure for deactivations as well as some sort of queuing mechanism when grains start swapping to keep the amount of work which is executing in parallel rationalized.


Web Push Notifications

As time goes by, the web becomes more and more powerful. It’s been around for a while, but I think that with Chrome and Edge finally supporting PWA apps (and native apps starting to become mobile web apps), it is time to embrace this technology. It is time to create web pages that can push notifications directly to the desktop even if the browser is closed.

This works right now on any device running Microsoft Edge, as well as devices that run Google Chrome (Android devices and desktops running Chrome).

Architecturally, the Service Worker part of your website (running in the background) creates a subscription with a push notification service provider and then that “token” is sent to your application server, which will use it to send you a notification.

What I didn’t grasp, is that you the concept of the push notification service provider is not any server running some software running a specific protocol, but rather tied directly to the browser. For example, the ONLY push notification service provider if the website is being viewed within Google Chrome is FCM (Firebase), Mozilla or Microsoft Edge (Windows WNS). In fact, these are the same technologies that are used by real native apps on those platforms.

Soooo…

  1. The API to subscribe and get some sort of token to enable push notifications is defined by internet standards, so it is safe to use in any browser on any website.
  2. Each browser will implement those methods using its own push notification service provider
  3. Sending a push notification will be different depending on whom the push notification service provider is.

Here is a link to a functional demo that works everywhere : https://webpushdemo.azurewebsites.net/


Office365 Two Factor Authentication

Two-factor authentication is an important thing, we should all now that by now. For a long time, I’ve had it activated on my Microsoft Account and Google Account. Today I decided to turn it on for the modelon.net Office365 tenant, I went into the admin tools and turned it on. Easy enough.

As expected, I had to remove the account from Outlook and Windows 10 and re-add them. But surprise, Outlook 2016 didn’t want to connect. It seems there is a manual intervention that needed to be done.

I am no IT admin and no PowerShell guru, but following the articles here saved my life:

https://support.office.com/en-us/article/Enable-or-disable-modern-authentication-in-Exchange-Online-58018196-f918-49cd-8238-56f57f38d662

https://docs.microsoft.com/en-us/powershell/exchange/exchange-online/connect-to-exchange-online-powershell/mfa-connect-to-exchange-online-powershell?view=exchange-ps

By using the instructions in the second link, you get some sort of Office365 configured local PowerShell thing. Once you open that, all you have to do is these two commands to make everything work:

Connect-EXOPSSession -UserPrincipalName erik.renaud@modelon.net
Set-OrganizationConfig -OAuth2ClientProfileEnabled $true

VSTS – Build & Release for To-Do.Studio’s web site

So my startup To-Do.Studio is advancing, and we started getting rolling on our informational website.

The first thing was to create a “web site” project with Visual Studio, and get it into GIT.

As you can see, it’s just a bunch of HTML, CSS and stuff, with a small web.config so that we can host on IIS (we are using Azure App Services for this). The first thing which is abnormal though and unlike a normal ASP.NET project, I do not have a CSPROJ that describes my project, instead, I have a PUBLISHPROJ, and it gets ignored by GIT. So how did we get here ?

Since Visual Studio relies on things like MSBUILD to “publish” projects, it needs some sort of project file and in this case, Visual Studio created the PUBLISHPROJ the first time I created the publish profile, this allows me to publish from the command line and more.

Although the file is GIT ignore, I had to add it to GIT in order for the VSTS build system to be able to publish this project.

The other modification we had to do was to add to GIT a special file, in the folder of the solution called “after.ToDoStudio.StaticWebFront.sln.targets”. Turns out MSBUILD looks for these files automatically to include them in your build process, without the need to modify the solution file. This is what we put in there:

<!--?xml version="1.0" encoding="utf-8"?-->
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 <Target Name="Deploy Website" AfterTargets="Build">
 <Message Text="Starting Website deployment" Importance="high">
 </Message>
 <MSBuild Projects="$(MSBuildProjectDirectory)\ToDoStudio.StaticWebFront\website.publishproj" 
 BuildInParallel="true" /> 
 </Target>
</Project>

What this does is ensure that after the Build action (which by default will build nothing cause this is a project with nothing to build), the system will automatically invoke the publish action using the PUBLISHPROJ.

Now we were ready to build this thing within VSTS.

As you can see here, our build is pretty standard with defaults used everywhere:

The release pipeline is also pretty standard, nothing fancy here:

 


Podcasts – Mine and Others

Let me be honest – I hadn’t listened to podcasts in a while, so was pleasantly surprised when I was recording a podcast with Mario Cardinal and Guy Barrette on CosmosDB and, during the initial chitchat, Mario talks about a great podcast he started listening to called After On.

Here is a link to the podcast that I have recorded, it is an introduction into CosmosDB, a service within Microsoft Azure that is a NOSQL and built from the ground up to scale with the web. It is the kind of technology that could be used to build things like Facebook or Twitter.

As for After-On, this is a podcast I instantly became hooked too. The discussions are out of this world, on diverse subjects and very in depth. Please go listen and let me know which is your favorite episode.


Shrinking USB keys to save a old Windows Tablet

At the first Microsoft Build event, they gave attendees a Samsung tablet, made for Windows 8. I loved that tablet (other than the fact that it was heavy) because it had a built-in cellular modem.

But sadly, last week, the thing crashed and was going into a boot loop. The only way to revive it was to reinstall everything…

This was going to be a challenge because this particular model required a 4gb Fat32 USB key. Unfortunatly it is 2018 and the smallest key i found was 16 gigabytes…

So search the internet and i didn’t find an exact recipe, but i did try something that could work – so i gave it a try. Open DiskPart and issue the commands:

list disk
select disk 1 (in my case it was 1, it might be different for you !)
clean
create partition primary size=4160
active

That’s it, then i formatted it, copied the contents of this ISO of Windows 10 onto it and bingo, it booted and in thirty minutes everything was up and running.

To get back the key to 16gb, just run the same instructions without the size-4160 part.

And that’s how you shrink a usb key.


Azure AppService to FTP (not in Azure)

Oy,  I just spent a crazy week to learn that :

It is impossible for an AppService application to connect to a FTP server on the internet (in passive mode). The reason is that each Azure App-Service is assigned a pool of IP addresses for outgoing traffic and Azure is free to choose a new outgoing IP for each connection (the natting stuff) and FTP expects the data connection to be from the same ip as the control connection.

This said, when using sftp, the handshake is negociated with the connection that composes the control channel. When the second connection to transfer data is built, the ftp server receives the connection from a potentially different IP address, which is a 425 type error.

Took us 3 days to diagnose, one evening to write a basic http web api that allows to post and get files, a few hours to install it all and a full day to rewrite the code that worked with ftp…

This said, it was something that no one on the team could of had predicted, live and learn !


Screen scrapping

One thing I believe in is constant change, and constant learning, ideally one thing per day. Sometimes it’s learning how to cook the best eggs benedict ever for breakfast, or sometimes it’s helping out a friend with a special request.

Today I was asked by a colleague if I could help extract data from a web site. As an architect, the first thing I look at in the “code” is a clean separation of what is presentation and what is data. Obviously, I did not find that, which made me realise how frameworks which render html mixed with data are bad bad bad. Why can’t everything follow MVVM with some binding of some sort.

Anyways, we needed a solution and what we wipped up was screen scrapping.

My first attempt was to write a small html page that loads jquery, does an ajax call to hit the webpage we needed data from and then extract it from it’s DOM… Turns out it was an easy to execute but I was met an error : CORS headers not found for file://dlg/extractData.html. GRRRRR

New strategy !

I opened Chrome, did a search for screen scrapping extensions and behold I found this.

An extension that would allow me to navigate to any page, lookup how it is built, understand it’s usage of CSS selectors and voilà. Any page that reuses the same css selector to represent repeating data (as in a list) can be extracted to a json or csv file.

Well, thanks DL for getting me to learn something new today !


ASPNET Core websites, developping with SSL

While working on the backend for To-Do.Studio, we ran into a scenario where we needed to test on a developer’s machine with https (ssl) enabled.

note – we use Kestral for testing and not IISExpress.

First thing we did was to try and add a https url in the LaunchSettings.json file. That didn’t work 🙂

What we found was that had to configure Kestral manually and tell it which certificate to use. The code in our Program class looks like this :

public static IWebHost BuildWebHost(string[] args) =>
   WebHost.CreateDefaultBuilder(args)
     .UseApplicationInsights()
     .UseAzureAppServices()
#if DEBUG
     .UseKestrel(t => {
        t.Listen(IPAddress.Loopback, 55172);
        t.Listen(IPAddress.Loopback, 55000, o =>
        {
            o.UseHttps("localhostCertificate.pfx", "password");
        });
     })
#endif
    .UseStartup<Startup>()
    .Build();

But how do we get a certificate ? There are various ways but i didn’t feel like finding my win32 sdk as some instruction’s on the web… so i decided to use my Ubuntu WSL…

Two commands

erik@ErikLAPTOP:~$ openssl req -x509 -days 10000 -newkey rsa:2048 -keyout cert.pem -out cert.pem
erik@ErikLAPTOP:~$openssl pkcs12 -export -in cert.pem -inkey cert.pem -out cert.pfx

and i had a good looking self made certificate. The hardest part was to copy this cert.pfx file to a Windows directory so i could use it my code.

Voilà ! after modifying my LaunchSettings.json, i could test in either http or https mode !

"ToDoStudio.Server_http": {
 "commandName": "Project",
 "launchBrowser": true,
 "environmentVariables": {
 "ASPNETCORE_ENVIRONMENT": "Development"
 },
 "applicationUrl": "http://localhost:55172/"
 },
 "ToDoStudio.Server_https": {
 "commandName": "Project",
 "launchBrowser": true,
 "environmentVariables": {
 "ASPNETCORE_ENVIRONMENT": "Development"
 },
 "applicationUrl": "https://localhost:55000/"
 }

Windows 10 Redstone 4

Well, as you all know, windows 10 Fall Creators Update is out (internally named Redstone3) and those of you who are Windows Insiders have started getting bits Redstone 4, which should go out in the spring and doesn’t have a public name yet…

In case you are reading Microsoft, this is what i would like to see…

  1. With Windows 10 ARM devices coming out soon…
    • Make Cortana better, make her take more real estate on the screen and not just a small popup where i am always scrolling. Innovate in tablet scenarios.
    • Make Windows Explorer a bit better in tablet mode, maybe tabs
    • Make the tablet experience one or two steps better, i miss swiping from the left quickly to change apps from my Windows 8 RT tablet.
  2. Upgrade Skype
    • Make Skype UWP better, allow to login with Microsoft work account and offer Skype4Business functionality. I don’t want to have two apps doing the same thing. OneDrive managed to do it, you do it also. It would be ok if Skype4Business functionality was in both Skype and Teams.
    • Make SMS sync work for each platform without bugs with mms support.
  3. Finalize Andromeda and Continuum, make it awesome, any Windows 10 device displays itself in the proper way according to how it’s being used.
  4. Upgrade networking, i use a LTE dongle which i cannot even mark as a metered network, innovate in the space somehow to bring innovation in the way in which we connect to wifi networks, lte or share the internet. At the same time, please have an automatic VPN mode for public networks.
  5. Innovate in sharing and continuing work…
    • Cloud Clipboard (was suppose to be in RS3)
    • Windows TimeLine (was suppose to be in RS3)
    • OneNote and Edge on every platform where everything syncs magically
  6. Surprise everyone with Edge
    • make it support PWA and all modern features, you are almost there
    • make it a rock hard stable, you are almost there
    • offer a few more innovations, you can do it !
  7. OneNote UWP should become a premier tool, deprecate the win32 version and make it ubiquitous
    1. Writing on webpages in Edge should not be a Edge feature but a feature of OneNote. It should work in any context over any app
    2. The Ink Workspace should revolve around OneNote
    3. Make OneNote accept personal and work accounts for synching like OneDrive
    4. Get rid of sticky notes,  OneNote should be my sticky notes app
  8. Make up your mind with the UWP office apps (Excel, Word, outlook, powerpoint). Stop having two flavours and be bold. I would prefer the UWP versions because sometimes less is more.
  9. Have a family mode which works on the lock screen or in a guest mode – i see this as a part of Andromeda and Continuum when the device is locked or we don’t’ know who is using the device, allow to browse the web in a inprivate session. There might be something called family apps in the store that are specialised to run without a user being logged in.
  10. Have a dongle based on windows 10 arm that has a hdmi plug, wifi, Bluetooth and a few usb ports. It can be plugged into a monitor to offer a full desktop or miracast receiver experience. Plugged into a tv for a tv experience. In enterprise this could replace thin clients or chromecasts in boardrooms. It also has to be cheaply priced.
  11. Get your media story together, after dropping Groove subscriptions, i feel tv is the next thing. Partner instead of dropping, you need presence in this space.
  12. Office personal and Ofice365 should have deeper integrations into the OS. I feel i am always on the web for these services
  13. Upgrade WSL to have a built-in wayland display server
  14. Upgrade WSL to bring Android apps to Windows 10 all factors, if you can’t beat them join them. Apps should be downloaded from the Store. If there were three major android stores (Google, Amazon and Microsoft), i think we could start moving towards shared api and not the closed Google Play apis… Xamarin could maybe provide that api…
  15. Keep on fighting and have presence in the mobile and tablet form factor, continue the Window S concept for low price points (but always upgradable if necessary)