Setting the stage
Augmented Reality is an amazing concept with immense potential, yet its adoption remains relatively limited. Despite the rapid advancements in hardware and software, we rarely see AR being integrated into everyday applications. This raises the question: why hasn’t AR become more mainstream? While we can’t fully answer this from a philosophical standpoint, as developers, we have a general idea of the challenges that hinder its widespread use.
Flutter (in)compatible
Broken Packages
- augmented_reality_plugin
- ar_flutter_plugin
- ar_flutter_plugin_updated
- ar_flutter_plugin_2
- ar_location_view
The ar_location_view is the most basic plugin amongst them. It is labeled as an AR plugin but we have our doubts about it. All it does is open the camera (wow) and render cards that are “being looked at”, at the scale they need to be. All this without hooking into the available ARCore / ARKit for support.
The major problem with this plugin, the camera, where other plugins outright failed to run, this plugin was unable to properly render the camera view. Almost all edges in the view were pixelated and since the plugin hasn’t been updated since it’s creation five months ago we looked for another.
The original plugin ar_flutter_plugin has not been updated for over 2 years but all the guides that are available are made using that plugin. We had to jump through a lot of hoops to get this library working, but by adding other libraries it was just not possible to keep using this one.
The successor to that plugin was ar_flutter_plugin_updated which hasn’t received an update in the last five months. But still works with the latests Flutter versions.
Another successor to the orignal plugin is ar_flutter_plugin_2, this plugin was released during our development phase. From the moment we knew it was available we tried it. But to no avail. We did contact the author of this library in hopes of getting a quick fix. The author did update eventually and is still actively updating whenever he can.
But our time was running out and we needed to make a decision. So we decided on ar_flutter_plugin_updated since it was the only one that had worked with our other libraries.
Call From The Void
Even though we found a package, it barely scratches the surface of what’s possible with ARCore. We can’t comment on ARKit, as the compatibility layer package remains broken and poorly maintained. There’s still a significant gap to bridge when it comes to porting AR capabilities to multi-platform development tools like Flutter.
Part of our idea to work on a package ourselves comes from the fact that we had to develop our own calculation processes to map markers based on position, using functions like Haversine. While it’s not a dealbreaker to implement this manually, the contrast with other parts of a Flutter mobile app—where things feel much more streamlined—discourages many developers who lack the time or experience to tackle these challenges.
To put the scale of this issue into perspective: building a new compatibility layer would likely require a dedicated team working exclusively on the package within The Lab.
Perhaps this is a worthwhile direction for the future?
Simplifying AR: Taking the Pokémon GO Route... Or Not?
Finally, one of the routes we could have chosen before landing the current way we do now with the calculation processes based on point of view and altitude correction, was just doing it the way that Pokemon Go does it. Before opening up your Augmented Reality view just grab the device position and using a simple conversion within viewing distance of the user you dont have to worry about altitude or POVs. This is somewhat a smart approach if you see the popularity of the Pokemon Go app. Everyone is in awe of the fact that it just works, even though it is a small implementation.
But that just didnt work for us, mostly the fact that you should be able to just walk around, open the AR view and then look around you where there could be multiple marker anchors at very specific locations.
The Sky Is The Limit
Quite literally, because we had to solve a fundamental problem: how do you accurately place markers in an AR view? Since AR markers rely on detected planes, their placement is inherently limited by what the camera can see. This meant we needed a way to position markers not just on visible surfaces but also in 3D space relative to the user’s position.
To achieve this, we combined plane detection with algorithms like Haversine for distance calculations and altitude correction. By merging plane data with the device’s field of view and position, we can determine a highly precise altitude at which a marker should appear. This ensures that markers are placed accurately in augmented space, even when the detected planes don’t directly align with their real-world positions.
Quite literally, because we had to solve a fundamental problem: how do you accurately place markers in an AR view? Since AR markers rely on detected planes, their placement is inherently limited by what the camera can see. This meant we needed a way to position markers not just on visible surfaces but also in 3D space relative to the user’s position.
To achieve this, we combined plane detection with algorithms like Haversine for distance calculations and altitude correction. By merging plane data with the device’s field of view and position, we can determine a highly precise altitude at which a marker should appear. This ensures that markers are placed accurately in augmented space, even when the detected planes don’t directly align with their real-world positions.
To bridge the gap between geolocation data and AR coordinates, we implemented two core conversion functions. The first function, convertARToLatLng, translates an AR position (relative to the user) into latitude and longitude coordinates:
Map convertARToLatLng(
Vector3 position,
double userLat,
double userLng,
) {
const double earthRadius = 6371000; // in meters
final latOffset = (position.z * 10) / earthRadius;
final lngOffset =
(position.x * 10) / (earthRadius * cos(userLat * pi / 180));
final newLat = userLat + (latOffset * (180 / pi));
final newLng = userLng + (lngOffset * (180 / pi));
return {"latitude": newLat, "longitude": newLng};
}
This allows us to determine the real-world location of a marker after it has been placed in AR. Conversely, convertLatLngToAR converts a latitude and longitude into an AR-relative position, ensuring that markers are positioned correctly within the AR view:
Vector3 convertLatLngToAR(
double markerLat,
double markerLng,
double userLat,
double userLng, {
double markerElevation = 0.0,
double userElevation = 0.0,
}) {
const double earthRadius = 6371000; // in meters
final latDiff = (markerLat - userLat) * (pi / 180);
final lngDiff = (markerLng - userLng) * (pi / 180);
final x = lngDiff * earthRadius * cos(userLat * (pi / 180));
final z = latDiff * earthRadius;
final y = markerElevation - userElevation; // Use markerElevation directly
// Scale to keep the AR coordinate space manageable
return Vector3(x / 10, y / 10, z / 10);
}
By leveraging these conversions, we ensure that markers maintain positional accuracy in both the real and augmented worlds. This straightforward approach—taking latitude, longitude, and altitude and seamlessly integrating them into a 3D AR environment—sets our AR experience apart.
It unlocks a range of creative use cases, from navigation overlays to location-based experiences. The only real limitations? The user’s imagination… and perhaps whether they have a ladder handy if they need to detect planes at higher altitudes.
Conclusion
Augmented Reality holds incredible potential, but its adoption remains hindered by technical challenges and inconsistent platform support. This article was co-authored by Pierre Franck, a fellow team member who struggled with the adoption of the Augmented Reality packages within our Flutter app. Our experience highlighted the gaps in existing solutions, from broken compatibility layers to the need for custom geolocation-to-AR mapping.
Rather than settling for simplified approaches like Pokémon GO’s method, we took the extra step to ensure precise marker placement in 3D space. By leveraging algorithms like Haversine and altitude correction, we bridged the gap between geolocation and AR, creating a more accurate and immersive experience.
However, the road to mainstream AR adoption is still long. The lack of well-maintained cross-platform packages makes development unnecessarily complex, and true AR innovation requires dedicated efforts from both individual developers and larger communities. If AR is to become more than just a novelty, we need better tools, stronger compatibility, and a commitment to solving these fundamental issues.
The technology is there—the challenge is making it accessible.
