Challenge on demand

Difficulty balancing is never an easy task : too easy and the game gets boring, too hard, and most players won't enjoy it.
Worse than that, everyone is different and some enjoy hard core difficulty while others prefer a smooth ride.

 VS

Who else better than the player himself knows what he is ready to face?
That's where the concept of "Challenge on Demand" comes from.

The idea is to have an average difficulty on the critical path but offer the exciting spikes of extra optional challenges to those who want it :)

BIOSHOCK
One of the best idea of Bioshock is exactly that : Big Daddies are challenge on demand.
You explore a level, work on your objectives, cross the path of the dreadful big daddies, defeat splicers, collect ammo and plasmids...and wait...you cross the path of big daddies, you see them, but you don't have to fight them!

Of course killing big dadies will allow you to reach the little sisters and either harvest their adam or rescue them and there's no advantage to leave them alive.

But they key point here is that big daddies are not initially aggressive + they move around.
What it means is that the player decides WHEN and WHERE he'll attack.
You can wait to have more ammo, or just try to fight the beast in an environment that suits you well.
As a player, the choice is yours.

DO IT!
Adding challenge on demand in your level provides the following benefits

• Extra challenge for players who enjoy it while not breaking the critical path for others
• Extra player motivation : to succeed, you must be prepared. It's a non mandatory goal, but it's teasing the player...especially as you are not forced to do it :)
• A way to put emphasis on some interesting gameplay mechanics that didn't fit on the critical path for technical/creative/narrative reasons. (stealth, platform, puzzle, etc)
  You don't like it as a player? Just don't do it :)

Of course it doesn't have to be a boss, it all depends on your game and its gameplay.

OPEN WORLD

They are the perfect setup for it as the player is usually free to go when and where he wants, hence being able to come back to the challenge when he feels ready for it.

RPG and MMORPGs are good at that : you don't have to do group dungeons to reach max level, and you don't need run raids to play, but they are here at your disposal when you have the time and feel ready for it.

Just make sure that they are easy to spot and reach when the player decides he is ready to go for it. (map icon, special landmark, vfx,etc)

OPEN LEVELS

As long as you can backtrack and there are some large bubbles of gameplay available to the player at once, it will work well. 

But make sure that the points of no return (if any) are obvious so the player doesn't miss the opportunity to do the challenge inadvertently. 

LINEAR MISSIONS

In a linear / scripted mission, it can be more tricky to add challenge on demand, but it's still feasable.

Drop your challenge as a "super obvious" dormant threat on the critical path so the player can't miss it but might think twice before attempting it.

Or just build it on the side of the critical path making sure it's easy to notice.

Of course, It's best if you can replay the mission later on.

As a general conclusion, I'd like to add that as long as the player knows what he has to do and what he can do in the game, it's always better to give him more freedom than not enough :)