How to Solve It (With Raycasting)

https://alicegg.tech//2024/05/01/how-to-solve-it.html

In 1945, mathematician George Pólya released the book “How to solve it”. It aims at helping math teachers guide their students into solving abstract problems by asking the right questions. It has since had a large influence on math education and computer science, to the point of being mentioned by Marvin Minksy as a book that everyone should know.

In this post, I will try to see how we can use Pólya’s methodology to solve a concrete software engineering problem: rendering 3D objects (using Golang).

A blue cubic sponge

Understanding the problem

Before starting to solve the problem, we must make sure that we completely understand the problem. The first question to ask ourselves is What is the data?

The data is a 3D object. The object is made out of triangles. Each triangle is made out of 3 vertices (and a normal vector). Those objects are stored in .STL files. I will parse them with the hschendel/stl lib.

The second question, which is probably the most important is What is the unknown?. Or in programming terms, What should the program output?

Our program should output an image. An image is a 2D matrix of pixels, each pixel representing a color. The most common way of representing color is the RGBA model, which stands for Red, Green, Blue, and Alpha. In Golang, images can be represented using the image.Image data structure from the standard library.

The third question is What is the condition (linking the data to the output)?

The data gives us information about the space occupied by our 3D object. If the 3D object is in front of our pixel, this pixel should be in a different color. We will use the method known as “raycasting” which consists of sending a ray from each pixel, and checking what the ray hits.

Devise a plan

Now that we have understood our problem a little bit better, we should try to plan what our solution will look like. The most helpful question to come up with a solution is Do you know a related problem?

Raycasting constists of sending a “ray” for each pixel of our image. If this ray intersects with our 3D object, the pixel needs to be updated to a different color. Since our 3D object is made entirely out of triangle, a related problem would be Does a vector intersect with a triangle?

To solve this we can implement the Möller–Trumbore intersection algorithm. This algorithm transforms the above problem into two new questions Does the ray intersect with the triangle’s plane? and if yes, Does the ray-plane intersection lie outside the triangle?

This first question is simple to solve, the only way a vector doesn’t intersect with a plane is if the vector and plane are parallel. In that case, the dot product of the ray and the triangle’s normal vector would be zero, since the dot product of two perpendicular vectors is 0 and the normal vector is itself perpendicular to the triangle’s plane.

If the ray intersects with our triangle’s plane, then we can check if the intersection is inside the plane by switching to barycentric coordinates. Barycentric coordinates are a way to represent a point in a plane in relation to the vertices of the triangle. Each corner of the triangle will get the coordinates (0,0,1), (0,1,0) and (1,0,0). Any point outside of the triangle will get coordinates outside of the range [0,1].

A triangle with a point P at the coordinates (a1, a2, a3)
Visualizing barycentric coordinates

Now that we know an algorithm that can solve our main issue, we can come up with the outline of our program:

func MTintersect(ray, triangle) bool {
	if isParallel(ray, triangle) {
		return false
	}
	u , v := projectBaryocentric(vec3, triangle)
	return u > 0 && u < 1 && v > 0 && u + v < 1
}
func main () {
	solid := readSTL()
	image := newImage(width, height)
	for i := range width {
		for j := range height {
			image.Set(i, j, white)
			ray := castRay(i, j)
			for triangle := range solid.Triangles {
				ok := MTintersect(ray, triangle)
				if ok {
					image.set(i, j, blue)
				}
			}
		}
	}
	writePNG(image)
}

Carrying out the plan

This is the easy part. We just write the code.

The main suggestion that Pólya makes, is to check that every step of the solution is correct. While programming, this can be achieved by writing unit tests to ensure the correctness of our code.

Looking back

Once we have something that seems to work it is tempting to just git push and call it a day. But there are a few more questions we should ask ourselves.

First Can we check the result?

A good way to answer that is to test our program ourselves, either by manually going through a checklist or by writing an integration test that covers our problem.

A book, a frog, a boat and the eiffel tower
Results of rendering a few .stl files

Then we should ask ourselves Can we derive the result differently?

This question is not only a good way to learn about other ways to solve our problem (like Scanline rendering in our case) but also a good opportunity to check if maybe the code we wrote was not the most intuitive solution and could be refactored.

The last question is Can you use the result for another problem?

We can answer this question by checking if our code is written in a way that is reusable enough if we ever want to. For example, the raycaster above could be used as the first step into the implementation of a more sophisticated ray tracing algorithm, if we wanted to handle reflections and lightning.

Conclusion

If you want to check the source code for the raycaster I made before writing this article, it is on my GitHub.

You can find How to solve it by Pólya in any good library.

To learn more about computer graphics check out Ray Tracing in a weekend. And for the details of the Möller-Trumbore algorithm, this video is the one that made the most sense to me.

{
"by": "zer0tonin",
"descendants": 2,
"id": 40220769,
"kids": [
40235296
],
"score": 54,
"time": 1714551258,
"title": "How to Solve It (With Raycasting)",
"type": "story",
"url": "https://alicegg.tech//2024/05/01/how-to-solve-it.html"
}
{
"author": null,
"date": null,
"description": "In 1945, mathematician George Pólya released the book “How to solve it”.It aims at helping math teachers guide their students into solving abstract problems…",
"image": "https://www.alicegg.tech/assets/2024-05-01-how-to-solve-it/sponge.jpg",
"logo": "https://logo.clearbit.com/alicegg.tech",
"publisher": "Alice GG",
"title": "Alice GG • How to solve it (with raycasting)",
"url": "https://www.alicegg.tech/2024/05/01/how-to-solve-it.html"
}
{
"url": "https://alicegg.tech//2024/05/01/how-to-solve-it.html",
"title": "Alice GG • How to solve it (with raycasting)",
"description": "In 1945, mathematician George Pólya released the book “How to solve it”. It aims at helping math teachers guide their students into solving abstract problems by asking the right questions. It has since had a...",
"links": [
"https://www.alicegg.tech//2024/05/01/how-to-solve-it.html",
"https://alicegg.tech//2024/05/01/how-to-solve-it.html"
],
"image": "https://www.alicegg.tech/assets/2024-05-01-how-to-solve-it/sponge.jpg",
"content": "<article>\n\t<div>\n\t\t<p>In 1945, mathematician George Pólya released the book “How to solve it”.\nIt aims at helping math teachers guide their students into solving abstract problems by asking the right questions.\nIt has since had a large influence on math education and computer science, to the point of being mentioned by Marvin Minksy as a book that <a target=\"_blank\" href=\"https://web.media.mit.edu/~minsky/papers/steps.html\">everyone should know</a>.</p>\n<p>In this post, I will try to see how we can use Pólya’s methodology to solve a concrete software engineering problem: rendering 3D objects (using Golang).</p>\n<figure>\n\t<img src=\"https://alicegg.tech/assets/2024-05-01-how-to-solve-it/sponge.jpg\" alt=\"A blue cubic sponge\" />\n</figure>\n<h2 id=\"understanding-the-problem\">Understanding the problem</h2>\n<p>Before starting to solve the problem, we must make sure that we completely understand the problem. The first question to ask ourselves is <strong>What is the data?</strong></p>\n<p>The data is a 3D object. The object is made out of triangles. Each triangle is made out of 3 vertices (and a normal vector).\nThose objects are stored in .STL files. I will parse them with the <a target=\"_blank\" href=\"https://pkg.go.dev/github.com/hschendel/stl\">hschendel/stl</a> lib.</p>\n<p>The second question, which is probably the most important is <strong>What is the unknown?</strong>. Or in programming terms, <strong>What should the program output?</strong></p>\n<p>Our program should output an image. An image is a 2D matrix of pixels, each pixel representing a color.\nThe most common way of representing color is the <a target=\"_blank\" href=\"https://en.wikipedia.org/wiki/RGBA_color_model\">RGBA model</a>, which stands for Red, Green, Blue, and Alpha.\nIn Golang, images can be represented using the <code>image.Image</code> data structure from the <a target=\"_blank\" href=\"https://pkg.go.dev/image#Image\">standard library</a>.</p>\n<p>The third question is <strong>What is the condition (linking the data to the output)?</strong></p>\n<p>The data gives us information about the space occupied by our 3D object. If the 3D object is in front of our pixel, this pixel should be in a different color.\nWe will use the method known as “raycasting” which consists of sending a ray from each pixel, and checking what the ray hits.</p>\n<h2 id=\"devise-a-plan\">Devise a plan</h2>\n<p>Now that we have understood our problem a little bit better, we should try to plan what our solution will look like.\nThe most helpful question to come up with a solution is <strong>Do you know a related problem?</strong></p>\n<p>Raycasting constists of sending a “ray” for each pixel of our image.\nIf this ray intersects with our 3D object, the pixel needs to be updated to a different color.\nSince our 3D object is made entirely out of triangle, a related problem would be <em>Does a vector intersect with a triangle?</em></p>\n<p>To solve this we can implement the <a target=\"_blank\" href=\"https://en.wikipedia.org/wiki/M%C3%B6ller%E2%80%93Trumbore_intersection_algorithm\">Möller–Trumbore intersection algorithm</a>.\nThis algorithm transforms the above problem into two new questions <em>Does the ray intersect with the triangle’s plane?</em> and if yes, <em>Does the ray-plane intersection lie outside the triangle?</em></p>\n<p>This first question is simple to solve, the only way a vector doesn’t intersect with a plane is if the vector and plane are parallel.\nIn that case, the dot product of the ray and the triangle’s normal vector would be zero, since the dot product of two perpendicular vectors is 0 and the normal vector is itself perpendicular to the triangle’s plane.</p>\n<p>If the ray intersects with our triangle’s plane, then we can check if the intersection is inside the plane by switching to barycentric coordinates.\nBarycentric coordinates are a way to represent a point in a plane in relation to the vertices of the triangle.\nEach corner of the triangle will get the coordinates (0,0,1), (0,1,0) and (1,0,0).\nAny point outside of the triangle will get coordinates outside of the range [0,1].</p>\n<figure>\n\t<img src=\"https://alicegg.tech/assets/2024-05-01-how-to-solve-it/barycentric.jpg\" alt=\"A triangle with a point P at the coordinates (a1, a2, a3)\" />\n\t<figcaption>\n\t\tVisualizing barycentric coordinates\n\t</figcaption>\n</figure>\n<p>Now that we know an algorithm that can solve our main issue, we can come up with the outline of our program:</p>\n<div><pre><code><span>func</span><span> </span><span>MTintersect</span><span>(</span><span>ray</span><span>,</span><span> </span><span>triangle</span><span>)</span><span> </span><span>bool</span><span> </span><span>{</span><span>\n\t</span><span>if</span><span> </span><span>isParallel</span><span>(</span><span>ray</span><span>,</span><span> </span><span>triangle</span><span>)</span><span> </span><span>{</span><span>\n\t\t</span><span>return</span><span> </span><span>false</span><span>\n\t</span><span>}</span><span>\n\t</span><span>u</span><span> </span><span>,</span><span> </span><span>v</span><span> </span><span>:=</span><span> </span><span>projectBaryocentric</span><span>(</span><span>vec3</span><span>,</span><span> </span><span>triangle</span><span>)</span><span>\n\t</span><span>return</span><span> </span><span>u</span><span> </span><span>&gt;</span><span> </span><span>0</span><span> </span><span>&amp;&amp;</span><span> </span><span>u</span><span> </span><span>&lt;</span><span> </span><span>1</span><span> </span><span>&amp;&amp;</span><span> </span><span>v</span><span> </span><span>&gt;</span><span> </span><span>0</span><span> </span><span>&amp;&amp;</span><span> </span><span>u</span><span> </span><span>+</span><span> </span><span>v</span><span> </span><span>&lt;</span><span> </span><span>1</span><span>\n</span><span>}</span><span>\n</span><span>func</span><span> </span><span>main</span><span> </span><span>()</span><span> </span><span>{</span><span>\n\t</span><span>solid</span><span> </span><span>:=</span><span> </span><span>readSTL</span><span>()</span><span>\n\t</span><span>image</span><span> </span><span>:=</span><span> </span><span>newImage</span><span>(</span><span>width</span><span>,</span><span> </span><span>height</span><span>)</span><span>\n\t</span><span>for</span><span> </span><span>i</span><span> </span><span>:=</span><span> </span><span>range</span><span> </span><span>width</span><span> </span><span>{</span><span>\n\t\t</span><span>for</span><span> </span><span>j</span><span> </span><span>:=</span><span> </span><span>range</span><span> </span><span>height</span><span> </span><span>{</span><span>\n\t\t\t</span><span>image</span><span>.</span><span>Set</span><span>(</span><span>i</span><span>,</span><span> </span><span>j</span><span>,</span><span> </span><span>white</span><span>)</span><span>\n\t\t\t</span><span>ray</span><span> </span><span>:=</span><span> </span><span>castRay</span><span>(</span><span>i</span><span>,</span><span> </span><span>j</span><span>)</span><span>\n\t\t\t</span><span>for</span><span> </span><span>triangle</span><span> </span><span>:=</span><span> </span><span>range</span><span> </span><span>solid</span><span>.</span><span>Triangles</span><span> </span><span>{</span><span>\n\t\t\t\t</span><span>ok</span><span> </span><span>:=</span><span> </span><span>MTintersect</span><span>(</span><span>ray</span><span>,</span><span> </span><span>triangle</span><span>)</span><span>\n\t\t\t\t</span><span>if</span><span> </span><span>ok</span><span> </span><span>{</span><span>\n\t\t\t\t\t</span><span>image</span><span>.</span><span>set</span><span>(</span><span>i</span><span>,</span><span> </span><span>j</span><span>,</span><span> </span><span>blue</span><span>)</span><span>\n\t\t\t\t</span><span>}</span><span>\n\t\t\t</span><span>}</span><span>\n\t\t</span><span>}</span><span>\n\t</span><span>}</span><span>\n\t</span><span>writePNG</span><span>(</span><span>image</span><span>)</span><span>\n</span><span>}</span><span>\n</span></code></pre></div>\n<h2 id=\"carrying-out-the-plan\">Carrying out the plan</h2>\n<p>This is the easy part. We just write the code.</p>\n<p>The main suggestion that Pólya makes, is to check that every step of the solution is correct.\nWhile programming, this can be achieved by writing unit tests to ensure the correctness of our code.</p>\n<h2 id=\"looking-back\">Looking back</h2>\n<p>Once we have something that seems to work it is tempting to just <code>git push</code> and call it a day.\nBut there are a few more questions we should ask ourselves.</p>\n<p>First <strong>Can we check the result?</strong></p>\n<p>A good way to answer that is to test our program ourselves, either by manually going through a checklist or by writing an integration test that covers our problem.</p>\n<figure>\n\t<img src=\"https://alicegg.tech/assets/2024-05-01-how-to-solve-it/image-grid.jpg\" alt=\"A book, a frog, a boat and the eiffel tower\" />\n\t<figcaption>\n\t\tResults of rendering a few .stl files\n\t</figcaption>\n</figure>\n<p>Then we should ask ourselves <strong>Can we derive the result differently?</strong></p>\n<p>This question is not only a good way to learn about other ways to solve our problem (like <a target=\"_blank\" href=\"https://en.wikipedia.org/wiki/Scanline_rendering\">Scanline rendering</a> in our case)\nbut also a good opportunity to check if maybe the code we wrote was not the most intuitive solution and could be refactored.</p>\n<p>The last question is <strong>Can you use the result for another problem?</strong></p>\n<p>We can answer this question by checking if our code is written in a way that is reusable enough if we ever want to.\nFor example, the raycaster above could be used as the first step into the implementation of a more sophisticated ray tracing algorithm, if we wanted to handle reflections and lightning.</p>\n<h2 id=\"conclusion\">Conclusion</h2>\n<p>If you want to check the source code for the raycaster I made before writing this article, it is on <a target=\"_blank\" href=\"https://github.com/zer0tonin/raycaster\">my GitHub</a>.</p>\n<p>You can find <em>How to solve it</em> by Pólya in any good library.</p>\n<p>To learn more about computer graphics check out <a target=\"_blank\" href=\"https://raytracing.github.io/books/RayTracingInOneWeekend.html\">Ray Tracing in a weekend</a>.\nAnd for the details of the Möller-Trumbore algorithm, <a target=\"_blank\" href=\"https://www.youtube.com/watch?v=fK1RPmF_zjQ\">this video</a> is the one that made the most sense to me.</p>\n\t</div>\n</article>",
"author": "",
"favicon": "https://alicegg.tech/favicon.png",
"source": "alicegg.tech",
"published": "",
"ttr": 199,
"type": ""
}