Simple rule to follow: Games exist via challenges, challenges are created by rule sets and those rule sets create limitations. When faced with a goal, the limitation = the challenge and thus over coming the said challenge results in an emotional response we tie to enjoyment.
For level design, one has to be player centric…player centric would imply knowing who your target audience is and where they are starting from. Within a level, you generally want to teach the player basic mechanics via challenges, whether its its a simple jump over a chasm or encountering a simple enemy. You need to introduce them to these challenges but the key is how you escalate the said challenges. Ideally you want to follow the flow chart, which is:
Translation of this pic is as follows: Too much challenge all at once (when they are not guided up to higher challenges via escalation of said challenges) will result in anxiety, which means negative emotional response and thus alienation. If theres no challenge (too easy, no surprise) you will get boredom, which is another form of alienation. Meaning you havent pulled out the emotional response required to immerse the player.
The happy spot is right in the middle and as the challenge goes up, and the complexity with player ability rises, if the player is in the sweet middle spot, they will have the best emotional response.
The other part is presentation and or story. In order for the player to care about their success in the game, you must find ways to pull out an emotional response. Really everything about movie enjoyment and game enjoyment comes right back to the behavioral sciences. Think about all those movies that start off with some tragic event to create the “problem” or challenge the player’s/viewer’s emotions. Imagine starting a game and you are presented with a cute puppy, aww its a puppy… everyone loves puppys…then all of a sudden a giant bird comes down and eats it. “wtf right?” already we have something to not only surprise the player (emotional response) but also something they can relate to (puppys). Now imagine right after that you see more puppys and you show the player more giant birds in the sky. Well obviously the connection is made, without telling the player what to do they interact with the game scenario you present. Their brain is working, birds = bad, puppies = good, goal = prevent birds from getting to puppies. Now you just have to show them how, this can be a gun or any other means to help the player achieve that goal. But wait, whats the challenge that kind of gets in the way? Maybe is ammo count…thus you create a resource management challenge.
Its up to you as the designer to find interesting ways to escalate that challenge while not making it impossible nor by making it too hard right off the bat…escalating challenges. Maybe you start off with 3 puppies and 5 birds and only 10 rounds of ammo. Then you are faced with 5 puppies 15 birds and 20 rounds of ammo…but wait, now maybe there are puppy eating gophers that pop out of the ground. Ok so you are presented with a bat to hit the gophers. Now you have a challenge mechanic that requires the player to identify, process (choose course of action) and react.
Level design pretty much takes the same approach. Challenges are presented in an escalating fashion, visuals should immerse as oppose to alienate the player. Think about what they player must identify and react upon. Know the tricks such as guiding players via light or subtle cues…let them come to their own conclusion without being told what to do or where to go (even if you subconciously guide them) because figuring out on “their own” is what gives a positive emotional response.
For your second issue. You need whats known as a GDD/LDD. Game design document and level design document. The point is that while you might create some mechanics that might be fun, they kind of only exist as a concept that needs to fit within a larger design document. The GDD will create the story/setting, it sets the tone, it describes the game play in great detail from start to finish, it states the target audience and within writing explains everything you need to know about the game and what the player SHOULD experience. The level design doc is broken up per level, how it fits within the larger GDD and has its own set of escalating challenges. The theme will fit the story created within the GDD. Many level designers will block out the levels, either on paper or within a game engine, they will begin to define the challenges and then flesh it out with the help of the environment/prop artist.
Its rarely ever a process of just “winging it”. You have to go in with a clear concept and design, then create the level via blocking volumes and pathways, add in the challenges and then make it look pretty.