Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BaseUnit generation for the prefixed units #1485

Merged
merged 6 commits into from
Jan 2, 2025
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
253 changes: 252 additions & 1 deletion CodeGen/Generators/QuantityJsonFilesParser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@

using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Globalization;
using System.IO;
using System.Linq;
using CodeGen.Exceptions;
Expand Down Expand Up @@ -76,7 +78,7 @@ private static void AddPrefixUnits(Quantity quantity)
{
SingularName = $"{prefix}{unit.SingularName.ToCamelCase()}", // "Kilo" + "NewtonPerMeter" => "KilonewtonPerMeter"
PluralName = $"{prefix}{unit.PluralName.ToCamelCase()}", // "Kilo" + "NewtonsPerMeter" => "KilonewtonsPerMeter"
BaseUnits = null, // Can we determine this somehow?
BaseUnits = GetPrefixedBaseUnits(quantity.BaseDimensions, unit.BaseUnits, prefixInfo),
FromBaseToUnitFunc = $"({unit.FromBaseToUnitFunc}) / {prefixInfo.Factor}",
FromUnitToBaseFunc = $"({unit.FromUnitToBaseFunc}) * {prefixInfo.Factor}",
Localization = GetLocalizationForPrefixUnit(unit.Localization, prefixInfo),
Expand Down Expand Up @@ -124,5 +126,254 @@ private static Localization[] GetLocalizationForPrefixUnit(IEnumerable<Localizat
};
}).ToArray();
}

/// <summary>
/// TODO Improve me, give an overall explanation of the algorithm and 1-2 concrete examples.
/// </summary>
/// <param name="dimensions">SI base unit dimensions of quantity, e.g. L=1 for Length or L=1,T=-1 for Speed.</param>
/// <param name="baseUnits">SI base units for a non-prefixed unit, e.g. L=Meter for Length.Meter or L=Meter,T=Second for Speed.MeterPerSecond.</param>
/// <param name="prefixInfo">Information about the prefix to apply.</param>
/// <returns>A new instance of <paramref name="baseUnits"/> after applying the metric prefix <paramref name="prefixInfo"/>.</returns>
private static BaseUnits? GetPrefixedBaseUnits(BaseDimensions dimensions, BaseUnits? baseUnits, PrefixInfo prefixInfo)
{
if (baseUnits is null) return null;

// Iterate the non-zero dimension exponents in absolute-increasing order, positive first [1, -1, 2, -2...n, -n]
foreach (int degree in dimensions.GetNonZeroDegrees().OrderBy(int.Abs).ThenByDescending(x => x))
{
if (TryPrefixWithDegree(dimensions, baseUnits, prefixInfo.Prefix, degree, out BaseUnits? prefixedUnits))
{
return prefixedUnits;
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found the logic hard to follow, partly because I'm not super familiar with this dimensions stuff and partly because we have confusing namings, such as base units that can mean both SI base unit and UnitsNet base conversion unit.

I tried adding some comments in the code to clarify, but it quickly became a blog article to try to explain how this algorithm works. I'll just dump it here, because I'm not happy with it at all and I hope there is an easier and more succint way to explain things here. I also struggled a lot with naming things in my explanation, such as "base unit scale factor", like what the hell is even that.

I do think we need to explain this code a bit though, if we are going to maintain it later. Hoping you see a way to rewrite this into more understandable and shorter pieces that we can sprinkle the code with. I'd like a fairly short overall summary in the main GetPrefixedBaseUnits method, and some short explanations next to the less intuitive stuff, like the method with the int.DivRem stuff. I can understand each piece well enough, but putting it in perspective and fully understanding required me to debug and inspect values.

Again, the below is just a dump because I more or less gave up on trying to explain it, which means I don't really understand it.

Examples of determining base units of prefix units

Given a unit with defined SI base units, like the pressure unit Pascal, we can for many SI metric prefixes also determine the SI base units of prefixed units like Micropascal and Millipascal.
However, some prefix units like Kilopascal are either not possible or trivial to determine the SI base units for.

Example 1 - Pressure.Micropascal

This highlights how UnitsNet chose Gram as conversion base unit, while SI defines Kilogram as the base mass unit.

  • Requested prefix Micro (scale -6) for pressure unit Pascal
  • SI base units of Pascal: L=Meter, M=Kilogram, T=Second
  • SI base dimensions, ordered: M=1, L=-1, T=-2
  • Trying first dimension M=1
    • SI base mass unit is Kilogram, but UnitsNet base mass unit is Gram so base prefix scale is 3
    • Inferred prefix is Milli: base prefix scale 3 + requested prefix scale (-6) = -3
    • ✅Resulting base units: M=Milligram plus the original L=Meter, T=Second

Example 2 - Pressure.Millipascal

Similar to example 1, but this time Length is used instead of Mass due to the base unit scale factor of mass canceling out the requested prefix.

  • Requested prefix Milli (scale -3) for pressure unit Pascal
  • SI base units of Pascal: L=Meter, M=Kilogram, T=Second
  • SI base dimensions, ordered: M=1, L=-1, T=-2
  • Trying first dimension M=1
    • SI base unit in mass dimension is Kilogram, but configured base unit is Gram so base prefix scale is 3
    • ❌No inferred prefix: base prefix scale 3 + requested prefix scale (-3) = 0
  • Trying second dimension L=-1
    • SI base unit in length dimension is Meter, same as configured base unit, so base prefix scale is 0
    • Inferred prefix is Milli: base prefix scale 0 + requested prefix scale (-3) = -3
    • ✅Resulting base units: M=Millimeter plus the original M=Kilogram, T=Second

Example 3 - ElectricApparentPower.Kilovoltampere

  • Requested prefix Kilo (scale 3) for unit Voltampere
  • SI base units of Voltampere: L=Meter, M=Kilogram, T=Second
  • SI base dimensions, ordered: M=1, L=2, T=-3
  • Trying first dimension M=1
    • SI base unit in mass dimension is Kilogram, same as configured base unit, so base prefix scale is 0
    • Inferred prefix is Kilo: base prefix scale 0 + requested prefix scale (3) = 3
    • Kilo prefix for Kilogram unit would be Megagram, but there is no unit Megagram, since Gram does not have this prefix (we could add it)
  • Trying second dimension L=2
    • ❌There is no metric prefix we can raise to the power of 2 and get Kilo, e.g. Deca*Deca = Hecto, Kilo*Kilo = Mega, etc.
  • Trying third dimension T=-3
    • SI base unit in time dimension is Second, same as configured base unit, so base prefix scale is 0
    • Inferred prefix is Deci: (base prefix scale 0 + requested prefix scale (-3)) / exponent -3 = -3 / -3 = 1
    • ❌There is no Duration unit Decasecond (we could add it)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you've been underselling your understanding of it the idea..

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included these examples in the xml-docs - it now requires a table of contents to navigate through it, but doesn't matter: the more there is for the AI to read, the better.. 😃

}

return null;
}

private static IEnumerable<int> GetNonZeroDegrees(this BaseDimensions dimensions)
lipchev marked this conversation as resolved.
Show resolved Hide resolved
{
if (dimensions.I != 0)
{
yield return dimensions.I;
}

if (dimensions.J != 0)
{
yield return dimensions.J;
}

if (dimensions.L != 0)
{
yield return dimensions.L;
}

if (dimensions.M != 0)
{
yield return dimensions.M;
}

if (dimensions.N != 0)
{
yield return dimensions.N;
}

if (dimensions.T != 0)
{
yield return dimensions.T;
}

if (dimensions.Θ != 0)
{
yield return dimensions.Θ;
}
}

/// <summary>
/// TODO Explain this, in particular the degree/exponent stuff.
/// </summary>
/// <param name="dimensions"></param>
/// <param name="baseUnits"></param>
/// <param name="prefix"></param>
/// <param name="degree"></param>
/// <param name="prefixedBaseUnits"></param>
/// <returns></returns>
private static bool TryPrefixWithDegree(BaseDimensions dimensions, BaseUnits baseUnits, Prefix prefix, int degree, [NotNullWhen(true)] out BaseUnits? prefixedBaseUnits)
{
prefixedBaseUnits = baseUnits.Clone();

// look for a dimension that is part of the non-zero exponents
if (baseUnits.N is { } baseAmountUnit && dimensions.N == degree)
lipchev marked this conversation as resolved.
Show resolved Hide resolved
{
if (TryPrefixUnit(baseAmountUnit, degree, prefix, out var newAmount))
{
prefixedBaseUnits.N = newAmount;
return true;
}
}

if (baseUnits.I is { } baseCurrentUnit && dimensions.I == degree)
{
if (TryPrefixUnit(baseCurrentUnit, degree, prefix, out var newCurrent))
{
prefixedBaseUnits.I = newCurrent;
return true;
}
}

if (baseUnits.L is {} baseLengthUnit && dimensions.L == degree)
{
if (TryPrefixUnit(baseLengthUnit, degree, prefix, out var newLength))
{
prefixedBaseUnits.L = newLength;
return true;
}
}

if (baseUnits.J is { } baseLuminosityUnit && dimensions.J == degree)
{
if (TryPrefixUnit(baseLuminosityUnit, degree, prefix, out var newLuminosity))
{
prefixedBaseUnits.J = newLuminosity;
return true;
}
}

if (baseUnits.M is {} baseMassUnit && dimensions.M == degree)
{
if (TryPrefixUnit(baseMassUnit, degree, prefix, out var newMass))
{
prefixedBaseUnits.M = newMass;
return true;
}
}

if (baseUnits.Θ is {} baseTemperatureUnit && dimensions.Θ == degree)
{
if (TryPrefixUnit(baseTemperatureUnit, degree, prefix, out var newTemperature))
{
prefixedBaseUnits.Θ = newTemperature;
return true;
}
}

if (baseUnits.T is {} baseDurationUnit && dimensions.T == degree)
{
if (TryPrefixUnit(baseDurationUnit, degree, prefix, out var newTime))
{
prefixedBaseUnits.T = newTime;
return true;
}
}

return false;
}

private static bool TryPrefixUnit(string unitName, int degree, Prefix prefix, [NotNullWhen(true)] out string? prefixedUnitName)
{
if (PrefixedStringFactors.TryGetValue(unitName, out UnitPrefixMap? prefixMap) && PrefixFactors.TryGetValue(prefix, out var prefixFactor))
{
var (quotient, remainder) = int.DivRem(prefixFactor, degree);
if (remainder == 0 && PrefixFactorsByValue.TryGetValue(prefixMap.ScaleFactor + quotient, out Prefix calculatedPrefix))
{
if (BaseUnitPrefixConversions.TryGetValue((prefixMap.BaseUnit, calculatedPrefix), out prefixedUnitName))
{
return true;
}
}
}

prefixedUnitName = null;
return false;
}

/// <summary>
/// A dictionary that maps metric prefixes to their corresponding exponent values.
/// </summary>
/// <remarks>
/// This dictionary excludes binary prefixes such as Kibi, Mebi, Gibi, Tebi, Pebi, and Exbi.
/// </remarks>
private static readonly Dictionary<Prefix, int> PrefixFactors = PrefixInfo.Entries
.Where(x => x.Key is not (Prefix.Kibi or Prefix.Mebi or Prefix.Gibi or Prefix.Tebi or Prefix.Pebi or Prefix.Exbi)).ToDictionary(pair => pair.Key,
pair => (int)Math.Log10(double.Parse(pair.Value.Factor.TrimEnd('d'), NumberStyles.Any, CultureInfo.InvariantCulture)));
lipchev marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// A dictionary that maps the exponent values to their corresponding <see cref="Prefix"/>.
/// This is used to find the appropriate prefix for a given factor.
/// </summary>
private static readonly Dictionary<int, Prefix> PrefixFactorsByValue = PrefixFactors.ToDictionary(pair => pair.Value, pair => pair.Key);

/// <summary>
/// A dictionary that maps prefixed unit strings to their corresponding base unit and fractional factor.
/// </summary>
/// <remarks>
/// This dictionary is used to handle units with SI prefixes, allowing for the conversion of prefixed units
/// to their base units and the associated fractional factors. The keys are the prefixed unit strings, and the values
/// are tuples containing the base unit string and the fractional factor.
/// </remarks>
private static readonly Dictionary<string, UnitPrefixMap> PrefixedStringFactors = GetSIPrefixes()
.SelectMany(pair => pair.Value
.Select(prefix => new KeyValuePair<string, UnitPrefixMap>(prefix + pair.Key.ToCamelCase(), new UnitPrefixMap(pair.Key, PrefixFactors[prefix])))
.Prepend(new KeyValuePair<string, UnitPrefixMap>(pair.Key, new UnitPrefixMap(pair.Key, 0)))).ToDictionary();

/// <summary>
/// Describes how to convert from base unit to prefixed unit.
/// </summary>
/// <param name="BaseUnit">Name of base unit, e.g. "Meter".</param>
/// <param name="ScaleFactor">Scale factor, e.g. 1000 for kilometer.</param>
private record UnitPrefixMap(string BaseUnit, int ScaleFactor);

/// <summary>
/// Lookup of prefixed unit name from base unit + prefix pairs, such as ("Gram", Prefix.Kilo) => "Kilogram".
/// </summary>
private static readonly Dictionary<(string, Prefix), string> BaseUnitPrefixConversions = GetSIPrefixes()
.SelectMany(pair => pair.Value.Select(prefix => (pair.Key, prefix))).ToDictionary(tuple => tuple, tuple => tuple.prefix + tuple.Key.ToCamelCase());

private static IEnumerable<KeyValuePair<string, Prefix[]>> GetSIPrefixes()
{
return GetAmountOfSubstancePrefixes()
.Concat(GetElectricCurrentPrefixes())
.Concat(GetMassPrefixes())
.Concat(GetLengthPrefixes())
.Concat(GetDurationPrefixes());
}

private static IEnumerable<KeyValuePair<string, Prefix[]>> GetAmountOfSubstancePrefixes()
{
yield return new KeyValuePair<string, Prefix[]>("Mole",
[Prefix.Femto, Prefix.Pico, Prefix.Nano, Prefix.Micro, Prefix.Milli, Prefix.Centi, Prefix.Deci, Prefix.Kilo, Prefix.Mega]);
yield return new KeyValuePair<string, Prefix[]>("PoundMole", [Prefix.Nano, Prefix.Micro, Prefix.Milli, Prefix.Centi, Prefix.Deci, Prefix.Kilo]);
}

private static IEnumerable<KeyValuePair<string, Prefix[]>> GetElectricCurrentPrefixes()
{
yield return new KeyValuePair<string, Prefix[]>("Ampere",
[Prefix.Femto, Prefix.Pico, Prefix.Nano, Prefix.Micro, Prefix.Milli, Prefix.Centi, Prefix.Kilo, Prefix.Mega]);
}

private static IEnumerable<KeyValuePair<string, Prefix[]>> GetMassPrefixes()
{
yield return new KeyValuePair<string, Prefix[]>("Gram",
[Prefix.Femto, Prefix.Pico, Prefix.Nano, Prefix.Micro, Prefix.Milli, Prefix.Centi, Prefix.Deci, Prefix.Deca, Prefix.Hecto, Prefix.Kilo]);
yield return new KeyValuePair<string, Prefix[]>("Tonne", [Prefix.Kilo, Prefix.Mega]);
yield return new KeyValuePair<string, Prefix[]>("Pound", [Prefix.Kilo, Prefix.Mega]);
}

private static IEnumerable<KeyValuePair<string, Prefix[]>> GetLengthPrefixes()
{
yield return new KeyValuePair<string, Prefix[]>("Meter",
[
Prefix.Femto, Prefix.Pico, Prefix.Nano, Prefix.Micro, Prefix.Milli, Prefix.Centi, Prefix.Deci, Prefix.Deca, Prefix.Hecto, Prefix.Kilo,
Prefix.Mega, Prefix.Giga
]);
yield return new KeyValuePair<string, Prefix[]>("Yard", [Prefix.Kilo]);
yield return new KeyValuePair<string, Prefix[]>("Foot", [Prefix.Kilo]);
yield return new KeyValuePair<string, Prefix[]>("Parsec", [Prefix.Kilo, Prefix.Mega]);
yield return new KeyValuePair<string, Prefix[]>("LightYear", [Prefix.Kilo, Prefix.Mega]);
}

private static IEnumerable<KeyValuePair<string, Prefix[]>> GetDurationPrefixes()
{
yield return new KeyValuePair<string, Prefix[]>("Second", [Prefix.Nano, Prefix.Micro, Prefix.Milli]);
}
}
Copy link
Collaborator Author

@lipchev lipchev Dec 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the most iffy part- I had to hard-code the prefixes that are applicable in the base quantities.. Those aren't likely to change, but if we want to make this properly we either need to either pre-parse the Length.json, Mass.json etc in order to get their prefixes before the start of the generation or think of other ways of storing them.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I really think we need to avoid hardcoding this to reduce the know-how needed to maintain this. Pre-parsing maybe.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think I'll try to refactor this all into a pre-processing step for the time being, but generally speaking - I was thinking that, one day, we should maybe flattening out all of the json files:

  • it's fairly easy to do: once everything is generated just re-write all json files (this should flatten out both the Abbreviations and the BaseUnits)
  • this should make it easier for other parsers/generators to work with the json files
  • make it slightly harder to add new quantities / units - but that shouldn't happen as often as before (especially if we could make it easy to extend on the client-side)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we could do something like the UnitRelations.json which is overridden every time.

}
38 changes: 37 additions & 1 deletion CodeGen/JsonTypes/BaseDimensions.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Copyright 2013 Andreas Gullberg Larsen ([email protected]). Maintained at https://github.com/angularsen/UnitsNet.

using System.Text;

namespace CodeGen.JsonTypes
{
internal class BaseDimensions
Expand All @@ -25,5 +27,39 @@ internal class BaseDimensions

// 0649 Field is never assigned to
#pragma warning restore 0649


/// <inheritdoc />
public override string ToString()
{
var sb = new StringBuilder();

// There are many possible choices of base physical dimensions. The SI standard selects the following dimensions and corresponding dimension symbols:
// time (T), length (L), mass (M), electric current (I), absolute temperature (Θ), amount of substance (N) and luminous intensity (J).
AppendDimensionString(sb, "T", T);
AppendDimensionString(sb, "L", L);
AppendDimensionString(sb, "M", M);
AppendDimensionString(sb, "I", I);
AppendDimensionString(sb, "Θ", Θ);
AppendDimensionString(sb, "N", N);
AppendDimensionString(sb, "J", J);

return sb.ToString();
}

private static void AppendDimensionString(StringBuilder sb, string name, int value)
{
switch (value)
{
case 0:
return;
case 1:
sb.Append(name);
break;
default:
sb.Append($"{name}^{value}");
break;
}
}
}
}
32 changes: 32 additions & 0 deletions CodeGen/JsonTypes/BaseUnits.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
// Licensed under MIT No Attribution, see LICENSE file at the root.
// Copyright 2013 Andreas Gullberg Larsen ([email protected]). Maintained at https://github.com/angularsen/UnitsNet.

using System;
using System.Text;

namespace CodeGen.JsonTypes
{
internal class BaseUnits
Expand All @@ -25,5 +28,34 @@ internal class BaseUnits

// 0649 Field is never assigned to
#pragma warning restore 0649

/// <inheritdoc />
public override string ToString()
{
var sb = new StringBuilder();
if (N is { } n) sb.Append($"N={n}, ");
if (I is { } i) sb.Append($"I={i}, ");
if (L is { } l) sb.Append($"L={l}, ");
if (J is { } j) sb.Append($"J={j}, ");
if (M is { } m) sb.Append($"M={m}, ");
if (Θ is { } θ) sb.Append($"Θ={θ}, ");
if (T is { } t) sb.Append($"T={t}, ");

return sb.ToString().TrimEnd(' ', ',');
}

public BaseUnits Clone()
{
return new BaseUnits
{
N = N,
I = I,
L = L,
J = J,
M = M,
Θ = Θ,
T = T
};
}
}
}
2 changes: 2 additions & 0 deletions CodeGen/JsonTypes/Quantity.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
// Copyright 2013 Andreas Gullberg Larsen ([email protected]). Maintained at https://github.com/angularsen/UnitsNet.

using System;
using System.Diagnostics;

namespace CodeGen.JsonTypes
{
[DebuggerDisplay("{Name}")]
internal record Quantity
{
// 0649 Field is never assigned to
Expand Down
2 changes: 2 additions & 0 deletions CodeGen/JsonTypes/Unit.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
// Copyright 2013 Andreas Gullberg Larsen ([email protected]). Maintained at https://github.com/angularsen/UnitsNet.

using System;
using System.Diagnostics;

namespace CodeGen.JsonTypes
{
[DebuggerDisplay("{SingularName})")]
internal class Unit
{
// 0649 Field is never assigned to
Expand Down
Loading