Alice, Bob and Mallory: Category C#

C# implicit string conversion

Jonas Elfström — Wed, 03 Feb 2010 17:32:00 +0100

I know how it works and I think I can see why but I'm still not very fond of how eager C# is to perform implicit string conversion.

Contrived example:

12	string s = -42 + '+' + "+" + -0.1 / -0.1 + "=" + (7 ^ 5) + " is " + true + " and not " + AddressFamily.Unknown;

s will be set to "1+1=2 is True and not Unknown"

The answer is in white text above, select the text to see it.

A more real problem is something like this

string str = 1 + 2 + "!=" + 1 + 2;

str will be set to "3!=12".

Edit 2010-02-08
This wouldn't be much of a problem if all objects in .NET always returned a decent string representation of their current state/value with ToString() but that's not the case. Instead "The default implementation returns the fully qualified name of the type of the Object.".
I don't like the inconsistency. It's way too late now but I think it would have been much better if only objects that really produces a human readable output of the data in the object should implement ToString(). If you want the name of the type of the Object there should be another way.

Finding primes in parallel

Jonas Elfström — Thu, 14 Jan 2010 22:55:00 +0100

Justin Etheredge has been blogging about his challenge to find prime numbers with LINQ. He later used AsParallel() (coming in .NET 4) to speed things up and then followed that up with a post about using The Sieve Of Eratosthenes.

As you can see in the comments of those posts I tried to speed the Sieve of Eratosthenes up by using Parallel.For in the inner loop. I also tried AsParallel() in the LINQ expression but it made no difference in either case. At most it got 5% faster. I'm not sure but it could be that because SoE is very memory intense we could have a scaling issue and maybe also memory bandwidth exhaustion. This is mere speculation.

I then searched for other algorithms and found The Sieve of Atkin. It uses less memory than SoE so I thought I'd give it a try.

I set the limit to 20,000,000 and then benchmarked it. It timed in on 2.48s so actually worse than the 2.2s that SoE took. Not good! Then I added Parallel.For in the loop that did most of the work and lo and behold, it scaled! I have two cores in my machine (T7200@2.0GHz) and the average runtime went down to 1.26s. That's almost linear and surprisingly good! If you happen have a quad core (or more) and feel like trying it out then please contact me. It would be interesting to see if it scales further.

static List<int> FindPrimesBySieveOfAtkins(int max)
{
    var isPrime = new BitArray((int)max+1, false);
    var sqrt = (int)Math.Sqrt(max);

    Parallel.For(1, sqrt, x =>
    {
        var xx = x * x;
        for (int y = 1; y <= sqrt; y++)
        {
            var yy = y * y;
            var n = 4 * xx + yy;
            if (n <= max && (n % 12 == 1 || n % 12 == 5))
                isPrime[n] = !isPrime[n];

            n = 3 * xx + yy;
            if (n <= max && n % 12 == 7)
                isPrime[n] = !isPrime[n];

            n = 3 * xx - yy;
            if (x > y && n <= max && n % 12 == 11)
                isPrime[n] = !isPrime[n];
        }
    });

    var primes = new List<int>() { 2, 3 };
    for (int n = 5; n <= sqrt; n++)
    {
        if (isPrime[n])
        {
            primes.Add(n);
            int nn = n * n;
            for (int k = nn; k <= max; k += nn)
                isPrime[k] = false;
        }
    }

    for (int n = sqrt + 1; n <= max; n++)
        if (isPrime[n])
            primes.Add(n);

    return primes;
}

This is C# 4.0 code, compiled in Visual C# 2010 Express Beta 2.

Edit 2010-01-20

Indications are that this does in fact not scale very good on a quad core. It's even worse, it seems it scales good on my old T7200 but not on a dual core E6320. I don't know why but of course the shared state of the isPrime BitArray is a huge problem and maybe it could be that differences in CPU architecture (FSB speed, caches and so on) in the E6320 is an explanation. Average execution time on the E6320 was 1290ms in a single thread and 1064ms in two.

If you want to try this in an older version of C# than 4.0 then check out this post.

A reader asked how I timed the executions. Here's how.

var steps = new List<long>();
var watch = new Stopwatch();

for (int i = 0; i < 10; i++) 
{
    watch.Reset();
    watch.Start();
    var primes = FindPrimesBySieveOfAtkins(20000000);
    watch.Stop();
    Console.WriteLine(watch.ElapsedMilliseconds.ToString());
    steps.Add(watch.ElapsedMilliseconds);
}
Console.WriteLine("Average: " + steps.Average().ToString());

Infinite ranges in C#

Jonas Elfström — Tue, 20 Oct 2009 20:41:00 +0200

I recently learned that C# is compliant with the IEEE 754-1985 for floating point arithmetics. That wasn't a big surprise but that division by zero is defined as Infinity in it was! It actually kind of bothers me that I didn't know this.

In mathematics division by zero is undefined for real numbers but I guess Infinity is a more pragmatic result. Or as a friend put it "IEEE stands for Institute of Electrical and Electronics Engineers not Institute of Mathematics"

1234	double n = 1.0;n = n / 0;if (n > 636413622384679305) System.Console.WriteLine("Yes it certainly is!");

This C# code does not throw an exception, it simply leaves n defined as Infinity and writes a line to the console.

Ruby is also IEEE 754-1985 compliant. It even lets you define infinite ranges.

Infinity=1.0/0
=>Infinity
(1..Infinity).include?(162259276829213363391578010288127)
=> true
(7..Infinity).step(7).take(3).inject(&:+) # 7+14+21
=> 42

I can't say I see very much use of this but it brings a kind of completeness to the handling of infinities. Unfortunately it seems we don't get that in C# out of the box because Enumerable.Range takes <int>,<int> as parameters and there's no Infinity definition for int. That's unless someone wrote a generic Range class. Turns out none other than Jon Skeet did in his MiscUtil. Download MiscUtil and then by using MiscUtil.Collections; you can:

double n = 1.0;
var infinity = n / 0;
var r = new Range<double>(0, infinity);
if (r.Contains(4711))
  System.Console.WriteLine("Yes it certainly does!");
var sum = r.Step(7.0).Take(3).Sum();

And guess what, it works like a charm! 4711 is part of positive infinity and sum is 42.0 and all is good.

Edit

There's also a couple of predefined constants. Thanks to Eric for pointing that out.

12	var r = new Range<double>(7, System.Double.PositiveInfinity);var sum = r.Step(7.0).Take(3).Sum();

The Thrush combinator in C#

Jonas Elfström — Tue, 06 Oct 2009 20:35:00 +0200

Last year I read Reg "Raganwald'" Braithwaite's excellent post The Thrush and he explains it as

The thrush is written Txy = yx. It reverses evaluation.

Back then I didn't even consider trying to implement it in C#. That was before I digged deeper into lambda expressions and extension methods in C# 3.0 and way before last night when I read Debasish Ghosh's post on how to implement the Thrush in Scala. After reading that my first thought was if it was possible to do the same in C#. Here's my attempt.

At first I struggled with the static typing and headed for an easy way out using Object in the extension method of Object:

12	public static object Into(this Object obj, Func<object, object> f){ return f.Invoke(obj); }

My goal was to translate the Ruby example

(1..100).select(&:odd?).inject(&:+).into { |x| x * x }

in Raganwald's post to C#.

Which reads "Take the numbers from 1 to 100, keep the odd ones, take the sum of those, and then answer the square of that number."

But with the Object based extension method I had to do some ugly casts.

var r = Enumerable.Range(1, 100).Where(x => Odd(x)).Sum().Into(x => (int)x * (int)x);

With som added typing I could do:

var result = Enumerable.Range(1, 100).Where(x => Odd(x)).Sum().Into(x => x * x);

But that merely moved the cast to the ext. method and also made it work for integers only.

12	public static int Into(this Object obj, Func<int, int> f){ return f.Invoke((int)obj); }

Then I remembered generics and method type inference.

12	public static T Into<T>(this T obj, Func<T, T> f){ return f(obj); }

And now not only the casts were gone but I also got a thrush combinator almost as flexible as the one in Ruby.

Contrived example follows:

12	var test = "ball";var ball = test.Into(s => "Are we having a " + s + " yet?");

The odd part

The Odd(x) method call in the calculation above is a plain static method.

12	private static bool Odd(int n){ return (n % 2 != 0); }

If you want an even more terse syntax you could try an ext. method on IEnumerable like this:

12	public static IEnumerable<int> Odd(this IEnumerable<int> en){ return en.Where(n => n % 2 != 0); }

Gives:

var result = Enumerable.Range(1, 100).Odd().Sum().Into(x => x * x);

Also as a general alternative to .Sum() I could have used .Aggregate((x, y) => x + y)) but I found it a bit verbose.

In C# I don't think it's possible to pull off the Symbol#to_proc stuff that Ruby does. That's the &: in the select(&:odd?) and the inject(&:+) in the Ruby example. Raganwald has a great post on that too.

Edit

Check out Jon Skeet's nice answer on StackOverflow to my question on how to make this even more Ruby-like. I have to try out that Operator class later though.

Edit 2009-10-07

One thing I found a bit surprising is that by implementing the Into ext. method in this way it not only works for all objects based on System.Object but it also works for value types.

1234	int n=4711;int oddOrZero = n.Into(x => x % 2 !=0 ? x : 0); // 4711n = 4712;oddOrZero = n.Into(x => x % 2 != 0 ? x : 0); // 0

Edit 2009-10-12

My confusion did stem from my lack of understanding of extension methods. Ex. methods are in fact not extending System.Object or any other type, they are "nothing more than a pleasant syntax for calling a static method" in case no instance method with the same name can be found.

No var for me in the foreach

Jonas Elfström — Tue, 09 Jun 2009 22:24:00 +0200

In C# 3.0 we got type inference or implicit typing as Microsoft likes to call it. As a Ruby programmer I've got a thing for essence over ceremony and those repetive declarations in C# (and Java) has always bothered me. So of course I quickly put var in my tool belt. If I want to create a certain object why should I have to state that twice?

1234	// C# 2.0Dictionary<Customer, List<PhoneNumber>> phonebook = new Dictionary<Customer, List<PhoneNumber>>(); // C# 3.0var phonebook = new Dictionary<Customer, List<PhoneNumber>>();

Still you should use it with care. I've seen:

12	var i = 5;var s = "This stmt is unprovable!";

And frankly, I do not agree.

A couple of days ago I almost thought I found a bug or limitation in the C# compiler. Something like the following would not compile:

String html = "<a href='http://is.gd/Uoip'>Recursion</a>,\r\n" +
              "see <a href='http://is.gd/Uoip'>recursion</a>.";
String links="";
var matches = Regex.Matches(html, "(a href=')(.*)('>)");
foreach (var match in matches) {
    links+=match.Groups[2]+"\r\n";
}

The compiler complained that Object had no Groups method. How come it could not see that Regex.Matches returned a MatchCollection and that that collection was populated with Match objects? Then it dawned on me. Back in the dark ages of C# 1.x we did not have generics. MatchCollection is an old class that implements ICollection and not ICollection(T) so the compiler could not infer the type. A quick change to:

foreach (Match match in matches) {

and we were good to go.

Get the value of and with using the null-coalescing operator

Jonas Elfström — Sat, 16 May 2009 00:38:00 +0200

I like the ?? operator that surfaced in C# 2.0. I'm often in environments where null values florish and ?? is a great way to handle them, especially for presentation. A while ago I found a use for the ?? operator in a way I'd never used it before.

The problem at hand was to return an account number from data that could be described as somewhat inconsistent.

The application had left room for the users to enter accounts in two by two ways. A customer could have multiple subsidiaries and in each subsidiary, by misconception, two different fields had been used as the account number. It was also possible to connect an account to all subsidairies of a customer at the same time as a specific subsidiary of that customer had an account defined.

Only if a unique account could be found it should be returned. All other cases should return a cause of failure so that the users could use that information to clean up the mess.

private static Account GetCustomerAccount(Customer customer)
{
  bool hasAccount = !String.IsNullOrEmpty(customer.AccountNo);
  bool hasAccountNoExtra = !String.IsNullOrEmpty(customer.AccountNoExtra);
  bool hasAllSubsAccount = !String.IsNullOrEmpty(customer.AccountNoAllSubs);
  bool hasAllSubsAccountNoExtra = 
       !String.IsNullOrEmpty(customer.AccountNoExtraAllSubs);
  var account = new Account();

  if ((hasAccount || hasAccountNoExtra) &&
      (hasAllSubsAccount || hasAllSubsAccountNoExtra)) 
  {
    account.Status = account.HasBothAllSubsAndSpecific;
    return account;
  }
  if ((hasAccount && hasAccountNoExtra) ||   
      (hasAllSubsAccount && hasAllSubsAccountNoExtra)) 
  {
    account.Status = account.HasBothAccountAndAccountExtra;
    return account;
  }

  account.AccountNo = customer.AccountNo ??
                      customer.AccountNoExtra ??
                      customer.AccountNoAllSubs ??
                      customer.AccountNoExtraAllSubs;

  if (account.AccountNo==null)
    account.Status=account.HasNoAccountDefined;

  return account;
}

Notice how AccountNo is set to the first non null value of the four. Imagine that with plain ifs. Here I believe the ??-operator both makes the code clear and saves us from a bunch of nested ifs.

Predicate<T> matches me

Jonas Elfström — Thu, 29 Jan 2009 21:29:00 +0100

I really like the Predicate(T) delegates that were added to the generic collections and lists in .NET 2.0. With the later addition of lambda expressions came cleanliness and readability.

Today we faced a quite simple problem that were made even simpler by the dear predicates. We had a kind of event log and wanted to filter it client side (Windows Forms) using a list of criterias. We began by implementing to filter by a number of categories. It ended up being only one row (in Visual Studio, for obvious reasons not here):

private List<Events> FilterEventsByCategory(List<Events> events,
                                        List<Category> categories) 
{
  return events.FindAll(event => 
      categories.Exists(category => category.CategoryId==event.CategoryId)); 
}

Neat!

Unit testing strains

Jonas Elfström — Mon, 05 May 2008 21:17:00 +0200

I've felt it and I've heard it from colleagues several times. Writing unit tests can be hard work. Especially adding unit test to an existing code base is, at best, cumbersome. Also it's one of those things with delayed gratification. Sometimes it's not even you that will benefit from them being there because the biggest win can be long down the road, when changes to the system has to be made.

Tests may seem to be isolated and it's even considered a good thing to keep them that way. Even so the tests of your application has a correlation to what the system aims to do on a bigger scale. This one of the things BDD focuses on. I think that one of the biggest advantages is that you in one process writes a specification and tests that ensures that the spec. is met. Testing becomes a natural part of the development process. This way it clearly shows that BDD and TDD are design processes and that it's certainly not all about adding unit tests.

Find out more about BDD on: http://behaviour-driven.org
It must be stressed that BDD is a rephrasing of existing good practice, it is not a radically new departure. Its aim is to bring together existing, well-established techniques under a common banner and with a consistent and unambiguous terminology.

For Ruby RSpec has almost become the de facto standard for BDD. The concepts Story, Scenario, and Test feels natural and the syntax is short and easy to read.

In languages like Java or C# the tests often becomes much more cluttered and some of that clutter is the extra code that comes with static typing. I believe that dynamically typed and overall dynamic languages like Ruby or Python could find a nice little niche here. They could become DSL's for testing.

RSpec is on it's way for .NET/C# via IronRuby and for Java via JRuby but don't hold your breath because they are still in alpha and beta.

.NET / C#
Testing .NET with IronRuby...
NSpecify => RSpec… well closer anyway

Java
Java Functional Testing with JRuby and RSpec
JtestR

Ruby
Slapp - A simple chat wall Merb tutorial. With nice exampes of using RSpec.
Behavior-driven testing with RSpec

ASP.NET MVC
ASP.NET MVC Test Framework Integration Walkthrough
MVC Preview - Testing
ASP.NET MVC Session at Mix08, TDD and MvcMockHelpers

What one-way hash function to use?

Jonas Elfström — Tue, 27 Feb 2007 17:01:00 +0100

One-way hash functions takes a message of any length as input and outputs a very large but fixed length number, called message digest or fingerprint. They can be used for "storing" passwords or as a signature that makes it possible to verify that you got the correct message.

MD5 got into problems over 10 years ago and SHA-1 could to be heading the same way. Until the new standard is published I would follow the crowd and recommend SHA-256.

Ruby

require 'digest/sha2'
quickfox="The quick brown fox jumps over the lazy dog"
Digest::SHA256.hexdigest(quickfox)

=> "d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592"

C#

using System.Security.Cryptography;

...

 ASCIIEncoding byteConverter = new ASCIIEncoding();
 string quickfox="The quick brown fox jumps over the lazy dog";
 HashAlgorithm sha256 = new SHA256Managed();
 byte[] hash = sha256.ComputeHash(byteConverter.GetBytes(quickfox));
 crypt.Text = Convert.ToBase64String(hash);