lol.... Hoarders! One of the smartest things you can do early is travel light and sell as much loot as you can right after every quest. Save up the money you earn to buy the house in Whiterun ASAP for storage ... The house is around 5000 I think and then maybe 2000 for the upgrades. Don't use it for storage until you get all the upgrades because it resets with upgrades. After that it's safe for storage and you can be more selective in selling vs saving. Also get a free nonmercenary Follower and they can help carry loot out of caves to sell - I picked up Sven from the Riverfall tavern.
In terms of what to pick up, you can immediately skip all food, wine, ale and books without skill perks. You might want to read the books but then drop them. I usually don't waste time looking in barrels or sacks either. Skip all misc junk like shovels, embalming tools, tankards, paper rolls, clothes ... Your house will give you a lot of that when you buy it. And if you ever need any of those things you can find some later. Keep a laptop nearby as you play, and you can quickly pause and google anything you're unsure about. Also keep an eye out for unusual or unique items you rarely see like empty skooma bottles - Those might be keepers.
Take as much as you can carry of everything else and sell it as soon as you fill up. If you hit some bandit camps while getting to a quest location, fast travel to a town and sell the loot before you go into the tunnel part (just make sure you discovered the quest location so you can fast travel back). To decide what to keep, you should plan early on what kind of character you want to have and get ruthless about not saving things that won't support your character. If you are a Mage type, you can sell warrior and stealth things. I'm a heavy armor player so I sell all light armor and mage things like staffs robes and scrolls that other players would keep. Enchanted items are better to save than non enchanted even if the non enchanted has a higher rating. Also keep all soul gems to recharge weapons. If you're not an archer take then sell all bows - they are lighter then other weapons to get back to the shops in town. For weapons keep only your very best few until you get a house. Keep all dragon bones and scales but these are heavy- another reason to get your house ASAP.
Ingredients are light so just keep what you can until you learn more about making potions. Anything with a sell value of +50 coins, sell until you get a house. Nirnroot, giants toe, Luna moth, chaurus egg, garlic, vampires dust, daedra heart, these are some keepers but most things you can sell. Sell any nonHealing potions also. Metal, I say sell until you get a house. You should upgrade loot weapons by smithing before you sell to earn more but the blacksmith will usually have ingots for sale so you don't have to cart them around. Here's a quick smith trick - gems like garnets and amethysts will make you a lot more money if you get some silver and quickly forge them into a necklace or ring before selling so silver is a keeper.
Carry around some cheap empty soul gems and get a weapon with a soul trap effect. Equip it whenever you are traveling on foot to new locations. Then when you kill any wild animals with it, the soul trap will fill your soul gems and you can recharge weapons with them.
