United have failed to win any of their four home league games this season, with the three defeats including a humiliating 6-1 rout against Tottenham
Pierre-Emerick Aubameyang condemned Manchester United to fresh misery at Old Trafford as the Arsenal forward's penalty clinched a 1-0 win, while Newcastle stopped Everton returning to the top of the Premier League with a 2-1 victory on Sunday.
Aubameyang ended a five-game goal drought in the league when he stepped up to convert a 69th-minute spot-kick after Paul Pogba fouled Hector Bellerin.
The Arsenal captain's first league goal since the opening weekend of the season gave the Gunners their first league win at Old Trafford since 2006.
Donny van de Beek nearly fashioned a late equaliser when his cross was scuffed by Mohamed Elneny into the face of Arsenal keeper Bernd Leno and on to the post.
But United have failed to win any of their four home league games this season, with the three defeats including a humiliating 6-1 rout against Tottenham.
It is United's worst start to a home league campaign since 1972-73, leaving Ole Gunnar Solskjaer's team languishing just six points above the relegation zone in 15th place.
After an encouraging end to last season when they secured a top-four finish, United have gone backwards and questions will be asked about Solskjaer's ability to hold on to his job if results do not improve soon.
Arsenal moved up to eighth place after ending a two-game losing run in the league.
Mikel Arteta's men were deserved winners, having dominated for long periods and hit the post through Willian in the first half.
Everton would have gone back above leaders Liverpool on goal difference with a win at St James' Park.
But, with James Rodriguez and Richarlison missing from the Everton team, Carlo Ancelotti's side succumbed to a limp defeat as Callum Wilson bagged his brace in the second half before Dominic Calvert-Lewin's late reply.
It was a difficult day for Everton, whose England keeper Jordan Pickford was dropped, losing his place to Robin Olsen after a turbulent start to the season.
Pickford has made a host of costly mistakes since Ancelotti took charge last season and was fortunate not to be sent off for the challenge which left Liverpool defender Virgil van Dijk needing knee surgery.
Newcastle took the lead in the 56th minute when Andre Gomes was adjudged to have fouled Wilson as he attempted to clear a corner at the near post.
It looked a harsh decision but Wilson stepped up to send Olsen the wrong way for his fifth goal of the season.
Wilson finished off Everton with a close-range effort from Ryan Fraser's deflected cross in the 84th minute.
Netting with a flicked finish, Calvert-Lewin's 11th club goal of the season came too late to stop Everton's second successive defeat.
They are without a win in three games and remain three points behind Liverpool in second place.
Elsewhere in Sunday's action, James Ward-Prowse celebrated his 26th birthday by scoring two fine free-kicks in Southampton's 4-3 win against Aston Villa.
Ralph Hasenhuttl's side lost their first two matches this term, but they have bounced back impressively and sit in third place after their fourth win in their last five games.
On the heels of a 3-0 defeat against Leeds, Villa's second successive loss served as a reality check after Dean Smith's men won their first four league games.
Southampton made their pressure pay in the 20th minute when Jannik Vestergaard headed in Ward-Prowse's free-kick.
Ward-Prowse curled a stunning free-kick past Villa keeper Emiliano Martinez from 25 yards in the 33rd minute, then netted another majestic free-kick from the edge of the area on the stroke of half-time.
Danny Ings bagged Southampton's fourth in the 58th minute, curling a superb shot into the top corner for his fifth club goal of the season.
Tyrone Mings reduced the deficit in the 62nd minute and Ollie Watkins converted a 90th-minute penalty after a foul on Grealish, who scored Villa's third with the last kick.
Tottenham host Brighton looking to climb into the top four in the final match of the day.