Vertical or Horizontal Side First

Interior lattice points of $S$ have both coordinates in $\{1,2,3\}$. A corner cannot be the first boundary point hit, because each neighbour of a corner, such as $(0,1)$ or $(1,0)$, is itself on the boundary. The walk is absorbed with probability $1$, since from any interior point some run of at most three steps in one direction reaches the boundary. So it stops on exactly one kind of side: vertical ($x\in\{0,4\}$) or horizontal ($y\in\{0,4\}$).

Symmetry on the diagonals. Reflection in the line $y=x$, $(x,y)\mapsto(y,x)$, maps $S$ to itself and swaps the vertical and horizontal sides. It also maps a symmetric walk to a walk with the same law, because it swaps up with right and down with left. Suppose the walk starts at a point on $y=x$. Then the reflected walk starts at the same point. Hence

The same argument with $(x,y)\mapsto(4-y,4-x)$ applies to starting points on the other diagonal $x+y=4$.

One step from $(1,2)$. Condition on the first step:

• left, to $(0,2)$: the walk stops on a vertical side;
• up, to $(1,3)$: this point lies on $x+y=4$;
• down, to $(1,1)$: this point lies on $y=x$;
• right, to $(2,2)$: this point lies on both diagonals.

By the Markov property, each of the last three cases gives a vertical side with probability $\tfrac12$. Therefore

Key idea. Every interior point on a diagonal is a fair split point, so you only need to follow the walk until it reaches a diagonal.