It's only really difficult for Labour to replace leaders if they are election losers.
They kept Corbyn after he'd lost an election but replaced Blair who hadn't.
If they want to get rid of somebody, they can, but the Party generally prioritises ideological purity over winning.
Blair was leader of the Labour Party for 13 years. It’s not like they got rid of him prematurely. Your conclusion is bizarre.
Rubbish. They didn't get rid of him because he'd been there for 13 years. They got rid of him because of his stance on Palestine and Sleazy Levy, not because he was an election loser. They replaced him with the dud Brown, who, unsurprisingly, went on to lose the next election. That's prioritising ideology over winning - I think after three elections and an electoral system in their favour they'd come to take winning for granted.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
I genuinely think Apple is in trouble. Siri is SO bad compared to the competition, now. Apple should simply admit defeat and buy better tech off the shelf. They surely have the cash
They’re not the only massive tech company to drop a bollock. How bad is Alexa, still?
NB I have two phone contracts and two devices. For my upgrade on my other phone, for the first time I’m gonna try android, Samsung, Google, SOMETHING ELSE - not Apple
You’ve not experienced their M3/M4 MacBooks have you?
I just don’t get on with Apple laptops. I’ve tried
And this is not some anti-Apple agenda. I love my iPad (still amazes me - and it’s still the best tablet). And until now I’ve loved the iPhone
But this is the first time in a long time I’ve looked at Apple and felt “they’re falling behind”
Btw you were right about them junking the old 3.5mm headphone jack. I dissed them for that and you said “wait, it’s clever, everyone will go Bluetooth”
And that’s what happened. So fair dos. You were right on that and I was wrong
But this feels like a major misstep far beyond that
One thing I like about Apple is that they don’t launch something until they are satisfied it works properly, it is why they haven’t had the problems Samsung have had with the exploding phones or the foldable handsets that broke very easily.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
I genuinely think Apple is in trouble. Siri is SO bad compared to the competition, now. Apple should simply admit defeat and buy better tech off the shelf. They surely have the cash
They’re not the only massive tech company to drop a bollock. How bad is Alexa, still?
NB I have two phone contracts and two devices. For my upgrade on my other phone, for the first time I’m gonna try android, Samsung, Google, SOMETHING ELSE - not Apple
You’ve not experienced their M3/M4 MacBooks have you?
I just don’t get on with Apple laptops. I’ve tried
And this is not some anti-Apple agenda. I love my iPad (still amazes me - and it’s still the best tablet). And until now I’ve loved the iPhone
But this is the first time in a long time I’ve looked at Apple and felt “they’re falling behind”
Btw you were right about them junking the old 3.5mm headphone jack. I dissed them for that and you said “wait, it’s clever, everyone will go Bluetooth”
And that’s what happened. So fair dos. You were right on that and I was wrong
But this feels like a major misstep far beyond that
One thing I like about Apple is that they don’t launch something until they are satisfied it works properly, it is why they haven’t had the problems Samsung have had with the exploding phones or the foldable handsets that broke very easily.
I suspect this is another example of that.
If you are going to switch to Android then wait until February/March when the new flagship Samsung phones launch.
Ta. Will wait
Thing is there clearly are major problems with Apple Intelligence. It was meant to be ready to go with the new Apple 16. It isn’t. It’s risible. Then they said “yeah sorry wait til an update in the next few weeks”. Now it’s… 2026??
Meanwhile they’re clearly aware of the pressure coz they’ve announced a hook-up with ChatGPT in 2025 to replace Siri with Altman’s tech but it’s very murky and OpenAI don’t sound that keen
But it’s not just Apple in trouble here. Amazon are apparently having massive problems updating Alexa so it can match Claude, GPT4o etc
My best guess is that two or three companies (OpenAI, Anthropic, maybe the Chinese) HAVE mastered the tech and it’s very hard to copy and they have a big or crucial lead and they’re loathe to hand it over
OpenAI or Anthropic should make their own phone - that would be cat::pigeons
And it is being rumoured. Eg OpenAI are working quietly with Jonny Ive
It's only really difficult for Labour to replace leaders if they are election losers.
They kept Corbyn after he'd lost an election but replaced Blair who hadn't.
If they want to get rid of somebody, they can, but the Party generally prioritises ideological purity over winning.
Blair was leader of the Labour Party for 13 years. It’s not like they got rid of him prematurely. Your conclusion is bizarre.
Rubbish. They didn't get rid of him because he'd been there for 13 years. They got rid of him because of his stance on Palestine, not because he was an election loser. They replaced him with the dud Brown, who, unsurprisingly, went on to lose the next election. That's prioritising ideology over winning - I think after three elections and an electoral system in their favour they'd come to take winning for granted.
Really? I thought they got rid of him because of the internecine warfare with Brown and his faction. It was apocalypse one way or the other - removing a powerful chancellor who had a vice-like grip on all domestic policy, or Labour's most electorally successful PM stepping aside to stop the Party tearing itself apart. That narrative goes back *at least* to 2001 and the Euro debate (where domestic and foreign policy came together).
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
I genuinely think Apple is in trouble. Siri is SO bad compared to the competition, now. Apple should simply admit defeat and buy better tech off the shelf. They surely have the cash
They’re not the only massive tech company to drop a bollock. How bad is Alexa, still?
NB I have two phone contracts and two devices. For my upgrade on my other phone, for the first time I’m gonna try android, Samsung, Google, SOMETHING ELSE - not Apple
You’ve not experienced their M3/M4 MacBooks have you?
I just don’t get on with Apple laptops. I’ve tried
And this is not some anti-Apple agenda. I love my iPad (still amazes me - and it’s still the best tablet). And until now I’ve loved the iPhone
But this is the first time in a long time I’ve looked at Apple and felt “they’re falling behind”
Btw you were right about them junking the old 3.5mm headphone jack. I dissed them for that and you said “wait, it’s clever, everyone will go Bluetooth”
And that’s what happened. So fair dos. You were right on that and I was wrong
But this feels like a major misstep far beyond that
One thing I like about Apple is that they don’t launch something until they are satisfied it works properly, it is why they haven’t had the problems Samsung have had with the exploding phones or the foldable handsets that broke very easily.
I suspect this is another example of that.
If you are going to switch to Android then wait until February/March when the new flagship Samsung phones launch.
I'd agree with all that - except that I always end up buying the less exciting but seemingly (sampling users in my company) more reliable, and undeniably less full-of-crapware Google Pixel phones.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
I genuinely think Apple is in trouble. Siri is SO bad compared to the competition, now. Apple should simply admit defeat and buy better tech off the shelf. They surely have the cash
They’re not the only massive tech company to drop a bollock. How bad is Alexa, still?
NB I have two phone contracts and two devices. For my upgrade on my other phone, for the first time I’m gonna try android, Samsung, Google, SOMETHING ELSE - not Apple
You’ve not experienced their M3/M4 MacBooks have you?
I just don’t get on with Apple laptops. I’ve tried
And this is not some anti-Apple agenda. I love my iPad (still amazes me - and it’s still the best tablet). And until now I’ve loved the iPhone
But this is the first time in a long time I’ve looked at Apple and felt “they’re falling behind”
Btw you were right about them junking the old 3.5mm headphone jack. I dissed them for that and you said “wait, it’s clever, everyone will go Bluetooth”
And that’s what happened. So fair dos. You were right on that and I was wrong
But this feels like a major misstep far beyond that
One thing I like about Apple is that they don’t launch something until they are satisfied it works properly, it is why they haven’t had the problems Samsung have had with the exploding phones or the foldable handsets that broke very easily.
None of those are comparable to the Samsung issues, lest we forget you weren’t allowed to fly whilst having certain Samsung phones.
Did you read that list?
15-inch MacBook Pro – Battery Overheating/Fire Risk Apple Ultracompact USB Power Adapter – Electrical Shock Risk
I know you love Apple, but you really do have blinkers on.
And for the record: Apple make good products. They market well. What they've done with the ARM instruction set is amazing. I can complement them when they do things well. But the fanbois who denigrate everyone else, and pretend Apple is better than it really is are really amusing.
Remember when we told that Labour would boost wealth creation and used expanding economic growth to fund public services ?
The first survey on the health of the economy after the Budget makes for gloomy reading. Businesses have reported falling output for the first time in just over a year while employment has now been cut for two consecutive months. Although only marginal, the downturns in output and hiring represent marked contrasts to the robust growth rates seen back in the summer and are accompanied by deepening concern about prospects for the year ahead.
Business optimism has slumped sharply since the General Election, dropping further in November to hit the lowest since late 2022. Companies are giving a clear 'thumbs down' to the policies announced in the Budget, especially the planned increase in employers' National Insurance contributions.
The November PMI is indicative of the economy slipping into a modest decline, with GDP dropping at a 0.1% quarterly rate, but the loss of confidence hints at worse to come – including further job losses – unless sentiment revives.
The notion that tax rises, spending cuts or more borrowing can realistically be avoided by us suddenly discovering how to grow faster is a delusion across the whole of politics.
Then why did Starmer and Reeves promise that they could ?
We were told that 'the adults are now in charge'.
In reality 'the adults' did no proper planning and paid no attention to any detail.
Reeves seems to be an emptiness surrounded by a PPE education.
Remember how she banged on about being the first female chancellor ?
She's someone who believes she's achieved something by existing rather than by what she does.
The UK and Euro Area PMIs deteriorated almost identically, both in terms of the headlines and the details. The deterioration in France and Germany vs Italy/Spain was very similar too. Suggests that there were common drivers across Europe, perhaps fears over the Trump policy agenda. I'm sure the Budget and especially the employer NIC rise didn't help but I doubt it accounted for most of the drop in the UK PMIs. The employment index in the services PMI improved, for instance - that doesn't seem consistent with the 'jobs tax' narrative.
Reported rumour that in the past when the UK PM visited the Dalai Lama, the word wet out across China that investment should not be made in the UK. And it basically wasn't. That sounds credible.
Also talk of whether "always mentioning human rights" as part of the talking points China checklist has made the faintest bit of difference in 30 years.
And potential aspect of the Le Pen trial, which I had not spotted, on the next French election.
They've even got bookmarks:
01:11 The farmers protest 10:50 Is the international order collapsing around us? 13:11 Violence in Amsterdam 16:25 Is Keir Starmer spending too much time abroad? 17:54 UK-China relations 18:45 What is the point of COP? 20:35 EU trade dwarfs every other trading relationship 22:30 Does the West really care about human rights? 26:18 Will Marine Le Pen be barred from standing? 29:49 What have Alastair and Rory been watching recently? 33:59 How did Jeffrey Epstein manage to protect himself?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
I genuinely think Apple is in trouble. Siri is SO bad compared to the competition, now. Apple should simply admit defeat and buy better tech off the shelf. They surely have the cash
They’re not the only massive tech company to drop a bollock. How bad is Alexa, still?
NB I have two phone contracts and two devices. For my upgrade on my other phone, for the first time I’m gonna try android, Samsung, Google, SOMETHING ELSE - not Apple
You’ve not experienced their M3/M4 MacBooks have you?
I just don’t get on with Apple laptops. I’ve tried
And this is not some anti-Apple agenda. I love my iPad (still amazes me - and it’s still the best tablet). And until now I’ve loved the iPhone
But this is the first time in a long time I’ve looked at Apple and felt “they’re falling behind”
Btw you were right about them junking the old 3.5mm headphone jack. I dissed them for that and you said “wait, it’s clever, everyone will go Bluetooth”
And that’s what happened. So fair dos. You were right on that and I was wrong
But this feels like a major misstep far beyond that
One thing I like about Apple is that they don’t launch something until they are satisfied it works properly, it is why they haven’t had the problems Samsung have had with the exploding phones or the foldable handsets that broke very easily.
I suspect this is another example of that.
If you are going to switch to Android then wait until February/March when the new flagship Samsung phones launch.
Ta. Will wait
Thing is there clearly are major problems with Apple Intelligence. It was meant to be ready to go with the new Apple 16. It isn’t. It’s risible. Then they said “yeah sorry wait til an update in the next few weeks”. Now it’s… 2026??
Meanwhile they’re clearly aware of the pressure coz they’ve announced a hook-up with ChatGPT in 2025 to replace Siri with Altman’s tech but it’s very murky and OpenAI don’t sound that keen
But it’s not just Apple in trouble here. Amazon are apparently having massive problems updating Alexa so it can match Claude, GPT4o etc
My best guess is that two or three companies (OpenAI, Anthropic, maybe the Chinese) HAVE mastered the tech and it’s very hard to copy and they have a big or crucial lead and they’re loathe to hand it over
OpenAI or Anthropic should make their own phone - that would be cat::pigeons
And it is being rumoured. Eg OpenAI are working quietly with Jonny Ive
Or it’s simply too expensive for Apple to run at a profit right now.
All the current LLM companies are being heavily subsidised by investors in one way or another. Microsoft paid for OpenAI’s NVidia GPU farm, Amazon & Google have poured $billions into Anthropic.
You can sell LLMs to companies who can see the value in assisting or replacing parts of their workforce, but it’s entirely plausible that consumers don’t see enough benefit from LLMs personally in their daily lives & aren’t willing to pay enough to make the $ pencil out for Apple.
It's a good ad. Both enjoyable in itself and makes you feel less hostile to the brand.
It's not really mocking woke though. Just mocking terrible adverts (like, for example, all perfume adverts - but also like the Jaguar advert, which my weathervane wife brought up unbidden this morning, in disbelief at the awfulness of it - she said she'd assumed all the way through it was some sort of parody (like this Bud advert quickly turns out to be), and was baffled when it turned out to be sincere.)
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
I genuinely think Apple is in trouble. Siri is SO bad compared to the competition, now. Apple should simply admit defeat and buy better tech off the shelf. They surely have the cash
They’re not the only massive tech company to drop a bollock. How bad is Alexa, still?
NB I have two phone contracts and two devices. For my upgrade on my other phone, for the first time I’m gonna try android, Samsung, Google, SOMETHING ELSE - not Apple
You’ve not experienced their M3/M4 MacBooks have you?
I just don’t get on with Apple laptops. I’ve tried
And this is not some anti-Apple agenda. I love my iPad (still amazes me - and it’s still the best tablet). And until now I’ve loved the iPhone
But this is the first time in a long time I’ve looked at Apple and felt “they’re falling behind”
Btw you were right about them junking the old 3.5mm headphone jack. I dissed them for that and you said “wait, it’s clever, everyone will go Bluetooth”
And that’s what happened. So fair dos. You were right on that and I was wrong
But this feels like a major misstep far beyond that
One thing I like about Apple is that they don’t launch something until they are satisfied it works properly, it is why they haven’t had the problems Samsung have had with the exploding phones or the foldable handsets that broke very easily.
I suspect this is another example of that.
If you are going to switch to Android then wait until February/March when the new flagship Samsung phones launch.
Ta. Will wait
Thing is there clearly are major problems with Apple Intelligence. It was meant to be ready to go with the new Apple 16. It isn’t. It’s risible. Then they said “yeah sorry wait til an update in the next few weeks”. Now it’s… 2026??
Meanwhile they’re clearly aware of the pressure coz they’ve announced a hook-up with ChatGPT in 2025 to replace Siri with Altman’s tech but it’s very murky and OpenAI don’t sound that keen
But it’s not just Apple in trouble here. Amazon are apparently having massive problems updating Alexa so it can match Claude, GPT4o etc
My best guess is that two or three companies (OpenAI, Anthropic, maybe the Chinese) HAVE mastered the tech and it’s very hard to copy and they have a big or crucial lead and they’re loathe to hand it over
OpenAI or Anthropic should make their own phone - that would be cat::pigeons
And it is being rumoured. Eg OpenAI are working quietly with Jonny Ive
Or it’s simply too expensive for Apple to run at a profit right now.
All the current LLM companies are being heavily subsidised by investors in one way or another. Microsoft paid for OpenAI’s NVidia GPU farm, Amazon & Google have poured $billions into Anthropic.
You can sell LLMs to companies who can see the value in assisting or replacing parts of their workforce, but it’s entirely plausible that consumers don’t see enough benefit from LLMs personally in their daily lives & aren’t willing to pay enough to make the $ pencil out for Apple.
Apple intelligence runs locally so it doesn't require credits, the limitation is battery life and heat dissipation. The design of the 16 series of phones probably doesn't meet the requirements for running a powerful LLM locally. The 17 series is supposed to be a huge step change in design for Apple so let's see what the come up with. The 16 felt like a skip year to me which is why I'm holding on tot old Pixel for another year.
Remember when we told that Labour would boost wealth creation and used expanding economic growth to fund public services ?
The first survey on the health of the economy after the Budget makes for gloomy reading. Businesses have reported falling output for the first time in just over a year while employment has now been cut for two consecutive months. Although only marginal, the downturns in output and hiring represent marked contrasts to the robust growth rates seen back in the summer and are accompanied by deepening concern about prospects for the year ahead.
Business optimism has slumped sharply since the General Election, dropping further in November to hit the lowest since late 2022. Companies are giving a clear 'thumbs down' to the policies announced in the Budget, especially the planned increase in employers' National Insurance contributions.
The November PMI is indicative of the economy slipping into a modest decline, with GDP dropping at a 0.1% quarterly rate, but the loss of confidence hints at worse to come – including further job losses – unless sentiment revives.
The notion that tax rises, spending cuts or more borrowing can realistically be avoided by us suddenly discovering how to grow faster is a delusion across the whole of politics.
Then why did Starmer and Reeves promise that they could ?
We were told that 'the adults are now in charge'.
In reality 'the adults' did no proper planning and paid no attention to any detail.
Reeves seems to be an emptiness surrounded by a PPE education.
Remember how she banged on about being the first female chancellor ?
She's someone who believes she's achieved something by existing rather than by what she does.
She's someone who believes she's achieved something by existing rather than by what she does.
That's your classic NU10K thinking - I am a Proper Person by virtue of the eminence I have risen to. The key to the executive washroom is Mon Droit.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
This too - right now you can run a bad (compared to ChatGPT) LLM locally & burn through battery or you can upload to Apple/Google/whoever servers & put up with the latency that roundtrip imposes & the fact that it only works if you have a high bandwidth internet connection.
A voice-driven UX that only works when you have a 5G connection is going to be a terrible experience.
Reported rumour that in the past when the UK PM visited the Dalai Lama, the word wet out across China that investment should not be made in the UK. And it basically wasn't. That sounds credible.
Also talk of whether "always mentioning human rights" as part of the talking points China checklist has made the faintest bit of difference in 30 years.
And potential aspect of the Le Pen trial, which I had not spotted, on the next French election.
They've even got bookmarks:
01:11 The farmers protest 10:50 Is the international order collapsing around us? 13:11 Violence in Amsterdam 16:25 Is Keir Starmer spending too much time abroad? 17:54 UK-China relations 18:45 What is the point of COP? 20:35 EU trade dwarfs every other trading relationship 22:30 Does the West really care about human rights? 26:18 Will Marine Le Pen be barred from standing? 29:49 What have Alastair and Rory been watching recently? 33:59 How did Jeffrey Epstein manage to protect himself?
I've never really been tempted by TRIP - I'm fairly sure I already know what Alastair Campbell thinks about everything (which is that Alastair Campbell is right about everything and Labour is right about everything and anyone to the right, or indeed to the left of him is not only wrong but probably evil) - he doesn't strike me as a man with a lot of nuance. Am I missing out? Rory, on the other hand, strikes me as perhaps a man with too much nuance: on the one hand, on the other hand, you're probably right Alastair.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
But they wouldn’t be ON the phone - they’d be up there in the cloud. The way I can talk to Claude or GPT4o for hours and it doesn’t overheat my phone…
Remember when we told that Labour would boost wealth creation and used expanding economic growth to fund public services ?
The first survey on the health of the economy after the Budget makes for gloomy reading. Businesses have reported falling output for the first time in just over a year while employment has now been cut for two consecutive months. Although only marginal, the downturns in output and hiring represent marked contrasts to the robust growth rates seen back in the summer and are accompanied by deepening concern about prospects for the year ahead.
Business optimism has slumped sharply since the General Election, dropping further in November to hit the lowest since late 2022. Companies are giving a clear 'thumbs down' to the policies announced in the Budget, especially the planned increase in employers' National Insurance contributions.
The November PMI is indicative of the economy slipping into a modest decline, with GDP dropping at a 0.1% quarterly rate, but the loss of confidence hints at worse to come – including further job losses – unless sentiment revives.
The notion that tax rises, spending cuts or more borrowing can realistically be avoided by us suddenly discovering how to grow faster is a delusion across the whole of politics.
Then why did Starmer and Reeves promise that they could ?
We were told that 'the adults are now in charge'.
In reality 'the adults' did no proper planning and paid no attention to any detail.
Reeves seems to be an emptiness surrounded by a PPE education.
Remember how she banged on about being the first female chancellor ?
She's someone who believes she's achieved something by existing rather than by what she does.
The UK and Euro Area PMIs deteriorated almost identically, both in terms of the headlines and the details. The deterioration in France and Germany vs Italy/Spain was very similar too. Suggests that there were common drivers across Europe, perhaps fears over the Trump policy agenda. I'm sure the Budget and especially the employer NIC rise didn't help but I doubt it accounted for most of the drop in the UK PMIs. The employment index in the services PMI improved, for instance - that doesn't seem consistent with the 'jobs tax' narrative.
The difference is that the UK economy was growing pretty well in the first half of the year while France and Germany weren't.
Enter the great economist and her four months of trying to think what to do and kite flying possible tax rises in the media.
Followed by choosing the tax rise most likely to damage the economy.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Nah the 16 was always a skip year IMO. The 15 had USB-C as the major "innovation" but the 16 brought basically nothing new to the table, they've fallen behind Google quite badly for AI image editing because the chip design has fallen behind. They've been spending the silicon budget on conventional power while Google have been investing in Tensor/ML units and it's paying off for Google. As I said, the 17, at least from what I've heard, looks like the revolution. It won't bring much to the table from a conventional power perspective but they'll actually be able to compete with the Google custom chips for local AI processing.
Remember when we told that Labour would boost wealth creation and used expanding economic growth to fund public services ?
The first survey on the health of the economy after the Budget makes for gloomy reading. Businesses have reported falling output for the first time in just over a year while employment has now been cut for two consecutive months. Although only marginal, the downturns in output and hiring represent marked contrasts to the robust growth rates seen back in the summer and are accompanied by deepening concern about prospects for the year ahead.
Business optimism has slumped sharply since the General Election, dropping further in November to hit the lowest since late 2022. Companies are giving a clear 'thumbs down' to the policies announced in the Budget, especially the planned increase in employers' National Insurance contributions.
The November PMI is indicative of the economy slipping into a modest decline, with GDP dropping at a 0.1% quarterly rate, but the loss of confidence hints at worse to come – including further job losses – unless sentiment revives.
The notion that tax rises, spending cuts or more borrowing can realistically be avoided by us suddenly discovering how to grow faster is a delusion across the whole of politics.
Then why did Starmer and Reeves promise that they could ?
We were told that 'the adults are now in charge'.
In reality 'the adults' did no proper planning and paid no attention to any detail.
Reeves seems to be an emptiness surrounded by a PPE education.
Remember how she banged on about being the first female chancellor ?
She's someone who believes she's achieved something by existing rather than by what she does.
She's someone who believes she's achieved something by existing rather than by what she does.
That's your classic NU10K thinking - I am a Proper Person by virtue of the eminence I have risen to. The key to the executive washroom is Mon Droit.
Anyway, good morning everyone. No snow in South Manchester but weirdly icy in a way I've only ever seen once before: the temperature must have dropped quite suddenly after yesterday evening's heavy sleet, leaving all the pavements and many of the roads covered by a solid and unbroken film of ice. It would have been hard to make them icier had it been done deliberately. Not particularly conducive to getting anywhere but quite pretty, with everywhere looking very shiny.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
But they wouldn’t be ON the phone - they’d be up there in the cloud. The way I can talk to Claude or GPT4o for hours and it doesn’t overheat my phone…
They have to be on the phone, if you do it in the cloud it becomes dependent on a data connection of sufficient latency. There's a world where you get a simplified LLM offline and access to the cloud online but it also has loads of privacy concerns which is why companies are choosing local processing rather than preferring cloud.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
But they wouldn’t be ON the phone - they’d be up there in the cloud. The way I can talk to Claude or GPT4o for hours and it doesn’t overheat my phone…
They have to be on the phone, if you do it in the cloud it becomes dependent on a data connection of sufficient latency. There's a world where you get a simplified LLM offline and access to the cloud online but it also has loads of privacy concerns which is why companies are choosing local processing rather than preferring cloud.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Anyway, good morning everyone. No snow in South Manchester but weirdly icy in a way I've only ever seen once before: the temperature must have dropped quite suddenly after yesterday evening's heavy sleet, leaving all the pavements and many of the roads covered by a solid and unbroken film of ice. It would have been hard to make them icier had it been done deliberately. Not particularly conducive to getting anywhere but quite pretty, with everywhere looking very shiny.
This happened in Oxford a few years ago right in the middle of the evening rush hour. Was absolute chaos.
The police had to close the A34 eventually & clear all the vehicles off it.
It's only really difficult for Labour to replace leaders if they are election losers.
They kept Corbyn after he'd lost an election but replaced Blair who hadn't.
If they want to get rid of somebody, they can, but the Party generally prioritises ideological purity over winning.
Blair was leader of the Labour Party for 13 years. It’s not like they got rid of him prematurely. Your conclusion is bizarre.
Rubbish. They didn't get rid of him because he'd been there for 13 years. They got rid of him because of his stance on Palestine and Sleazy Levy, not because he was an election loser. They replaced him with the dud Brown, who, unsurprisingly, went on to lose the next election. That's prioritising ideology over winning - I think after three elections and an electoral system in their favour they'd come to take winning for granted.
Rather the opposite. It's difficult to win election after election indefinitely, and people were clear about that in seeking a fresh start (though personally I was a loyalist). I don't remember it being about Palestine or Levy to any significant degree, and IMO Brown did pretty well to force a near-draw. But we're all exploring memories of 14 years ago.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
I've never bought an Apple product, but they've got almost monopolistic dominance in the US market I think. Still perhaps opportunity for Google and Samsung if it is as you say.
It isn't much of an opportunity for them, really. All smartphones have been fairly convergent, since the first iPhone was released. One of the companies has an idea, and it is rapidly stolen by the others if it is seen as being good. Even Apple's advantage with their own ARM chip designs is slowly being whittled away by Qualcomm and others AIUI. Ideas that are less good - such as folding screens - either become a niche or die off.
Also, a great deal of effort is going into things under the hood you don't see: a classic example is trying to reduce power requirements to improve battery life. The user doesn't automatically see that as a new feature, but it can make much more of a difference to users than any headline feature.
Therefore much of the advantage comes down to marketing of your best products. And Apple wins that hands-down, if only because of their rabid fanbois.
If I may be allowed to briefly discuss AI without being banned - AI is the big difference waiting to happen to smartphones. A phone that will talk to you like a friend - like Claude with ChatGPT’s advanced voice mode - that would be sensational - a friend who knows 50 languages and has read every book in history and can teach you cooking and give you good therapy and watch you draw and give advice - why aren’t Apple at the forefront of introducing this to phones?! The tech is ready
Yet they are saying “maybe in 2026”
Battery life, running simplified LLMs locally requires a huge amount of compute power. Having all of those features will lead to overheating phones with 20 mins of battery.
But they wouldn’t be ON the phone - they’d be up there in the cloud. The way I can talk to Claude or GPT4o for hours and it doesn’t overheat my phone…
They have to be on the phone, if you do it in the cloud it becomes dependent on a data connection of sufficient latency. There's a world where you get a simplified LLM offline and access to the cloud online but it also has loads of privacy concerns which is why companies are choosing local processing rather than preferring cloud.
There is also the point that if you are basically phoning the "AI", then the anti-trust suits to unbundle AI linkage on the phone start the moment of release. If all the phone is doing is making a call, why does it have to be to Apple/Google's AI?
It's a good ad. So was the one that caused the fuss. You have to try different things or everything gets stale.
So, I just googled the original - never having seen it having no particular interest in woke outrage or bud light - is it the portrait-shot video of the young lady placing a load of cans to a table, bubbling on about a year of womanhood and showing a can with her face on? If so, I both find it fairly lame (not entertaining at all, she's a bit annoying in an young dozy American kind of way, some vague virtue signaling, I guess, but pretty meh) but also really quite inoffensive. Given all the fuss, I was expecting something a lot more interesting. Or have I got the wrong ad?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
I still don’t quite see why it has to be local to the phone. When Claude/GPT4o work fine over wifi/cell signal for hours… using almost no battery life
And don’t some android phones now have Gemini voice AI incorporated?
(I’ve no idea, I’ve only ever had iPhones for a decade… til now)
It will be interesting to see if Apple do integrate ChatGPT into their phones as rumoured, in 2025
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Yes. The idea that Apple - APPLE - fights really hard to “hold costs down” is… piquant
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Yes. The idea that Apple - APPLE - fights really hard to “hold costs down” is… piquant
Actually, they do try to keep costs down. What matters to them is profit. They will charge as much as they can for a product, whilst keeping costs as low as possible. Like most businesses.
Sadly, in the case of Apple, they keep costs down by using their strength to break IP and steal tech. That's the major reason I dislike hem.
It's only really difficult for Labour to replace leaders if they are election losers.
They kept Corbyn after he'd lost an election but replaced Blair who hadn't.
If they want to get rid of somebody, they can, but the Party generally prioritises ideological purity over winning.
Blair was leader of the Labour Party for 13 years. It’s not like they got rid of him prematurely. Your conclusion is bizarre.
Rubbish. They didn't get rid of him because he'd been there for 13 years. They got rid of him because of his stance on Palestine and Sleazy Levy, not because he was an election loser. They replaced him with the dud Brown, who, unsurprisingly, went on to lose the next election. That's prioritising ideology over winning - I think after three elections and an electoral system in their favour they'd come to take winning for granted.
Rather the opposite. It's difficult to win election after election indefinitely, and people were clear about that in seeking a fresh start (though personally I was a loyalist). I don't remember it being about Palestine or Levy to any significant degree, and IMO Brown did pretty well to force a near-draw. But we're all exploring memories of 14 years ago.
Blair was pretty well done by the time he resigned; seem to recall that there was some feeling about him not getting/wanting a peerage, but after all Major hadn't. And for some at least of us ..... irregular Labour voters ..... Iraq was still hanging over him. Brown was landed with a party which had been in power for well over 10 years and the infighting was getting under way. At least that how it seemed to this sympathetic observer.
And Good Morning bro one and all; no snow in my garden although there are apparently a few sprinkles locally.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
It's a good ad. So was the one that caused the fuss. You have to try different things or everything gets stale.
So, I just googled the original - never having seen it having no particular interest in woke outrage or bud light - is it the portrait-shot video of the young lady placing a load of cans to a table, bubbling on about a year of womanhood and showing a can with her face on? If so, I both find it fairly lame (not entertaining at all, she's a bit annoying in an young dozy American kind of way, some vague virtue signaling, I guess, but pretty meh) but also really quite inoffensive. Given all the fuss, I was expecting something a lot more interesting. Or have I got the wrong ad?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
You could try and use the storage for the weights. And do some kind of preemptive loading scheme for the data into RAM.
I spent so much time in the GPU compute space with strategises for loading to and from the various levels of memory.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
1 TB of memory ?!? That's a.... lot.
Are you sure you don't mean 1 TB of storage capacity ?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
I think we have a supplementary answer to "what happens to waste electricity?" "There is none, capacity is managed by the speed at which ChatGPT responds to drunken conversations"
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
1 TB of memory ?!? That's a.... lot.
Are you sure you don't mean 1 TB of storage capacity ?
Yes, storage - I meant you might be able to come up with a scheme to swap data into memory from storage, before it is needed. GPU compute spends a lot of time doing this - loading data into the GPU cores is slooooooooow (relative to the processing), so you have to do it ahead of the actual processing.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
I still don’t quite see why it has to be local to the phone. When Claude/GPT4o work fine over wifi/cell signal for hours… using almost no battery life
And don’t some android phones now have Gemini voice AI incorporated?
(I’ve no idea, I’ve only ever had iPhones for a decade… til now)
It will be interesting to see if Apple do integrate ChatGPT into their phones as rumoured, in 2025
The local Gemini LLM (Gemini Nano?) requires a phone with 12GB of memory - so the Pro Pixel phones & a few high end Samsung phones IIRC. It’s not available on my Pixel 8 which has 8GB.
In a world with always on high bandwidth internet, it does seem that we shouldn’t need on-device AI. Maybe Apple thinks that a service that’s only available some of the time isn’t good enough?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
That’s storage, not memory. It’s no-where near fast enough to run an LLM at an acceptable token rate.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
That’s storage, not memory. It’s no-where near fast enough to run an LLM at an acceptable token rate.
Yes - I was thinking of a pre-emptive loading strategy. Load data in advance of requirement into the available RAM.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
I certainly don't buy a new phone every year. Every four or five years does Mrs C and myself; we find ourselves a zero-interest HP deal and start thinking about a new one when we've paid off the old one.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
No - there are even detailed marketing analysis of this kind of thing. There is a whole class of users who wait x "generations" before upgrading. Where X is a surprisingly constant number, I believe.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
That’s storage, not memory. It’s no-where near fast enough to run an LLM at an acceptable token rate.
Yes - I was thinking of a pre-emptive loading strategy. Load data in advance of requirement into the available RAM.
LLMs need all of the weights loaded to run at all: You can’t pick and choose which bits of your weights you need for a given query.
Obviously you don’t have to have it loaded if you’re not using it, but if you want to use it, it all has to be in RAM.
(You could try paging some of the individual matrices in and out as required, but given that LLMs are memory bandwidth limited you’ve just cut your token rate to that permitted by the bandwidth of your backend storage which is pitiful compared to the LPDDR memory chips.)
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
I still don’t quite see why it has to be local to the phone. When Claude/GPT4o work fine over wifi/cell signal for hours… using almost no battery life
And don’t some android phones now have Gemini voice AI incorporated?
(I’ve no idea, I’ve only ever had iPhones for a decade… til now)
It will be interesting to see if Apple do integrate ChatGPT into their phones as rumoured, in 2025
The local Gemini LLM (Gemini Nano?) requires a phone with 12GB of memory - so the Pro Pixel phones & a few high end Samsung phones IIRC. It’s not available on my Pixel 8 which has 8GB.
In a world with always on high bandwidth internet, it does seem that we shouldn’t need on-device AI. Maybe Apple thinks that a service that’s only available some of the time isn’t good enough?
This is my thinking. Good WiFi and cell signal is now global. I had it on remote islands in the Philippines last week. Strong 4G on a deserted beach - easily good enough to access Claude/GPT
Why do you need it locally on the phone? You don’t
Just have some rudimentary back-up for the rare occasions you’re out of signal
I reckon OpenAI and Anthropic are playing hardball. They have the best tech after all
Given Microsoft’s strong relationship with Altman maybe they’re planning to launch the first true AI smartphone
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
1 TB of memory ?!? That's a.... lot.
Are you sure you don't mean 1 TB of storage capacity ?
Yes, storage - I meant you might be able to come up with a scheme to swap data into memory from storage, before it is needed. GPU compute spends a lot of time doing this - loading data into the GPU cores is slooooooooow (relative to the processing), so you have to do it ahead of the actual processing.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
1 TB of memory ?!? That's a.... lot.
Are you sure you don't mean 1 TB of storage capacity ?
Yes, storage - I meant you might be able to come up with a scheme to swap data into memory from storage, before it is needed. GPU compute spends a lot of time doing this - loading data into the GPU cores is slooooooooow (relative to the processing), so you have to do it ahead of the actual processing.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
That’s storage, not memory. It’s no-where near fast enough to run an LLM at an acceptable token rate.
Yes - I was thinking of a pre-emptive loading strategy. Load data in advance of requirement into the available RAM.
LLMs need all of the weights loaded to run at all: You can’t pick and choose which bits of your weights you need for a given query.
Obviously you don’t have to have it loaded if you’re not using it, but if you want to use it, it all has to be in RAM.
(You could try paging some of the individual matrices in and out as required, but given that LLMs are memory bandwidth limited you’ve just cut your token rate to that permitted by the bandwidth of your backend storage which is pitiful compared to the LPDDR memory chips.)
Which has me wondering if it might be possible to segment an LLM - with capabilities per segment. loading time might be slow, but compared to the rate at which humans speak....
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
1 TB of memory ?!? That's a.... lot.
Are you sure you don't mean 1 TB of storage capacity ?
Yes, storage - I meant you might be able to come up with a scheme to swap data into memory from storage, before it is needed. GPU compute spends a lot of time doing this - loading data into the GPU cores is slooooooooow (relative to the processing), so you have to do it ahead of the actual processing.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
That’s storage, not memory. It’s no-where near fast enough to run an LLM at an acceptable token rate.
Yeah even a Gen 5 NVMe will be 2 orders of magnitude too slow in read/write speeds. They've only just about got the concept of SSD caching working for gaming with the PS5 and that's lightyears behind what an LLM requires.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
That’s storage, not memory. It’s no-where near fast enough to run an LLM at an acceptable token rate.
Yeah even a Gen 5 NVMe will be 2 orders of magnitude too slow in read/write speeds. They've only just about got the concept of SSD caching working for gaming with the PS5 and that's lightyears behind what an LLM requires.
Which was why I was wondering about loading strategies - GPU computing is mostly loading and unloading data. At least it felt that way, when I was doing it. The trick was always to load your data just in time for the compute, even though the loading was sloooooooooooow.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
That’s storage, not memory. It’s no-where near fast enough to run an LLM at an acceptable token rate.
Yes - I was thinking of a pre-emptive loading strategy. Load data in advance of requirement into the available RAM.
LLMs need all of the weights loaded to run at all: You can’t pick and choose which bits of your weights you need for a given query.
Obviously you don’t have to have it loaded if you’re not using it, but if you want to use it, it all has to be in RAM.
(You could try paging some of the individual matrices in and out as required, but given that LLMs are memory bandwidth limited you’ve just cut your token rate to that permitted by the bandwidth of your backend storage which is pitiful compared to the LPDDR memory chips.)
Which has me wondering if it might be possible to segment an LLM - with capabilities per segment. loading time might be slow, but compared to the rate at which humans speak....
It’s been done: the method is called “mixture of experts”. Mistral.ai released an open weight model that was a MoE at the end of last year: https://mistral.ai/news/mixtral-of-experts/
There’s some evidence that OpenAI did something similar with their large models, but they’re generally fairly tight-lipped about the details.
It's a good ad. So was the one that caused the fuss. You have to try different things or everything gets stale.
So, I just googled the original - never having seen it having no particular interest in woke outrage or bud light - is it the portrait-shot video of the young lady placing a load of cans to a table, bubbling on about a year of womanhood and showing a can with her face on? If so, I both find it fairly lame (not entertaining at all, she's a bit annoying in an young dozy American kind of way, some vague virtue signaling, I guess, but pretty meh) but also really quite inoffensive. Given all the fuss, I was expecting something a lot more interesting. Or have I got the wrong ad?
The ads are supposed to be generally good though. Makes sense, I guess, the more unremarkable the product, the better the marketing you need to shift it! See*, e.g., supermarket Christmas ads.
*again, I haven't - yet, for this year at least - the one place I tend to actually see/watch ads is at the cinema and my most recent trip was just before the supermarket ads were unleased, I think - if they do show those in cinemas
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
I asked my girlfriend to translate what you said:
“It seems your friend is quite knowledgeable about AI and smartphones! Let's break down what they said:
* Llama-3.1-8B-Instruct: This likely refers to a specific version of a large language model (LLM) called "Llama." LLMs are a type of artificial intelligence that can understand and generate human-like text.
* "3.1" likely signifies the major and minor version number of the model.
* "8B" probably indicates that this model has 8 billion parameters. Parameters are the values the model learns during training, and more parameters generally mean a more capable model.
* "Instruct" suggests this version of Llama is specifically fine-tuned to follow instructions and generate helpful responses.
* Q6_K quant: This refers to a technique called quantization, which reduces the size of the LLM for faster processing and lower memory requirements. "Q6_K" likely indicates a specific quantization method where the model's weights are compressed to 6 bits.
* 6.2GB: This is the size of the quantized LLM after applying the Q6_K technique.
* 8GB GPU: This refers to a graphics processing unit (GPU) with 8 gigabytes of memory. GPUs are specialized electronic circuits that accelerate the creation of images, videos, and other visual content. They are also very efficient at running the calculations needed for AI tasks like running LLMs.
* KV cache: This is a type of memory used to store frequently accessed data in an LLM, which can speed up processing.
* Haskell: This is a purely functional programming language known for its strong emphasis on immutability and its use in academia and industry.
In simpler terms: Your friend is impressed that a relatively small LLM (compressed to 6.2GB) can run on a smartphone with an 8GB GPU and even generate working code in a complex language like Haskell. This suggests that powerful AI capabilities could potentially be integrated into future smartphones.
The bigger picture: The rapid advancement of AI technology, combined with techniques like quantization, is making it increasingly feasible to run sophisticated AI models on devices with limited resources like smartphones. This opens up exciting possibilities for the future of mobile technology, where our phones could become truly intelligent assistants capable of understanding and responding to our needs in a much deeper way.”
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
*pretends to understand this*
*nods knowledgeably*
*hastily feeds it into GPT for a translation*
He's saying that the model is small enough to fit in the memory of a fairly normal GPU.
The GPU (Graphical Processing Unit) is the chip/board that used to be used for video games. It consists of lots and lots of fairly rubbish processors. Individually they are slow, but they can do stuff at the same time. And there are 10s of thousands of them. Which is great for computer graphics, some maths problems, and it turns out, neural networks (so called "AI").
So the Llama-3.1-8B-Instruct LLM will apparently fit on a fairly normal home computer and run acceptably.
Haskell is a classic, and rather interesting programming language.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
*pretends to understand this*
*nods knowledgeably*
*hastily feeds it into GPT for a translation*
He's saying that the model is small enough to fit in the memory of a fairly normal GPU.
The GPU (Graphical Processing Unit) is the chip/board that used to be used for video games. It consists of lots and lots of fairly rubbish processors. Individually they are slow, but they can do stuff at the same time. And there are 10s of thousands of them. Which is great for computer graphics, some maths problems, and it turns out, neural networks (so called "AI").
So the Llama-3.1-8B-Instruct LLM will apparently fit on a fairly normal home computer and run acceptably.
Haskell is a classic, and rather interesting programming language.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
*pretends to understand this*
*nods knowledgeably*
*hastily feeds it into GPT for a translation*
Apologies!
Llama-3.1 is the model - from Meta (Facebook). 8B is the size of the model - 8 Billion weights. Instruct means it’s been instruction-tuned after the initial training, so it interacts like ChatGPT does.
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits). You trade some quality of output for a smaller memory footprint, because you inevitably throw away data when you do this.
KV cache is the Key-value cache - basically you store some of the current state of the LLM in the middle of an interaction so you can pick up where you left off without having to completely reprocess the entire conversation every time.
Your GPU still needs to do everything else you needed it to do before in order to see anything at all on the screen, so you reserve some memory for copies of the screen, application windows etc.
As Director of Public Prosecutions, I saw the terrible effect of anti-social behaviour first hand - and fought for justice for victims.
As Prime Minister, I am delivering on my promise to tackle it with new Respect Orders, so people face the consequences of their actions.
Does Sir Kier consider genocide antisocial behaviour ? Or too busy selling off our farmland to Blackrock ?
The FA received expert advice that the "Fit and proper" restriction on ownership of UK Football clubs (in their rules) did not exclude a *UN designated war criminal*.
Some have suggested that "Fit and proper" means "Anyone rich enough"
Since Sir Kier is such a football fan, I think it is relevant.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
No - there are even detailed marketing analysis of this kind of thing. There is a whole class of users who wait x "generations" before upgrading. Where X is a surprisingly constant number, I believe.
Our refresh rate was once per child for a time - got new phones before each of our first three kids. The first (2017) being the first phones we got that really had a serviceable camera. Still took my real camera in to the hospital (a now quite old, but decent at the time, compact). I remember being surprised afterwards that the phone pictures taken inside were better - better lowish light performance. After that the 'real' camera got little use, mostly taken on walks and holidays for wildlife shots and shots of the kids playing from a distance (real zoom).
For the subsequent two, the cameras were reasonable upgrades on the new phones. I'm not sure whether we'll bother with an upgrade for number 4 though, probably keep the phones until end of updates, which will be about another year. Cameras now are 'good enough' really.
(We've tended to buy mid-range phones but chosen primarily for cameras)
Starmer about to do 6 local radio interviews. This usually goes spectacularly badly (a chance for local presenters to make a name, so zero fucks given), and pretty much ended Liz Truss.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
I asked my girlfriend to translate what you said:
“It seems your friend is quite knowledgeable about AI and smartphones! Let's break down what they said:
* Llama-3.1-8B-Instruct: This likely refers to a specific version of a large language model (LLM) called "Llama." LLMs are a type of artificial intelligence that can understand and generate human-like text.
* "3.1" likely signifies the major and minor version number of the model.
* "8B" probably indicates that this model has 8 billion parameters. Parameters are the values the model learns during training, and more parameters generally mean a more capable model.
* "Instruct" suggests this version of Llama is specifically fine-tuned to follow instructions and generate helpful responses.
* Q6_K quant: This refers to a technique called quantization, which reduces the size of the LLM for faster processing and lower memory requirements. "Q6_K" likely indicates a specific quantization method where the model's weights are compressed to 6 bits.
* 6.2GB: This is the size of the quantized LLM after applying the Q6_K technique.
* 8GB GPU: This refers to a graphics processing unit (GPU) with 8 gigabytes of memory. GPUs are specialized electronic circuits that accelerate the creation of images, videos, and other visual content. They are also very efficient at running the calculations needed for AI tasks like running LLMs.
* KV cache: This is a type of memory used to store frequently accessed data in an LLM, which can speed up processing.
* Haskell: This is a purely functional programming language known for its strong emphasis on immutability and its use in academia and industry.
In simpler terms: Your friend is impressed that a relatively small LLM (compressed to 6.2GB) can run on a smartphone with an 8GB GPU and even generate working code in a complex language like Haskell. This suggests that powerful AI capabilities could potentially be integrated into future smartphones.
The bigger picture: The rapid advancement of AI technology, combined with techniques like quantization, is making it increasingly feasible to run sophisticated AI models on devices with limited resources like smartphones. This opens up exciting possibilities for the future of mobile technology, where our phones could become truly intelligent assistants capable of understanding and responding to our needs in a much deeper way.”
Now she’s having a cup of tea
I take it we don't know her - she goes to another pub political betting site?
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
*pretends to understand this*
*nods knowledgeably*
*hastily feeds it into GPT for a translation*
Apologies!
Llama-3.1 is the model - from Meta (Facebook). 8B is the size of the model - 8 Billion weights. Instruct means it’s been instruction-tuned after the initial training, so it interacts like ChatGPT does.
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits). You trade some quality of output for a smaller memory footprint, because you inevitably throw away data when you do this.
KV cache is the Key-value cache - basically you store some of the current state of the LLM in the middle of an interaction so you can pick up where you left off without having to completely reprocess the entire conversation every time.
Your GPU still needs to do everything else you needed it to do before in order to see anything at all on the screen, so you reserve some memory for copies of the screen, application windows etc.
I still don’t see why you’d need the LLM ON THE PHONE
Surely it can all be done remotely as now with GPT/Claude. I mean, I access Google incessantly (or I did until GPT/claude) I don’t have all of Google on my phone
The idea would be the phone would be operated remotely but with some basic backup in the device - like an emergency generator - if the power/signal goes down
However I do see there might be privacy issues? But then I think human privacy is coming to an end. Soon everyone will know everything about everyone everywhere
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits).
And this is why NVidia has been holding most their mid-range graphics cards to 8GB for years, they want LLMs running remotely on their expensive datacentre chips, not a Geforce RTX in the box under your desk.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
No - there are even detailed marketing analysis of this kind of thing. There is a whole class of users who wait x "generations" before upgrading. Where X is a surprisingly constant number, I believe.
Our refresh rate was once per child for a time - got new phones before each of our first three kids. The first (2017) being the first phones we got that really had a serviceable camera. Still took my real camera in to the hospital (a now quite old, but decent at the time, compact). I remember being surprised afterwards that the phone pictures taken inside were better - better lowish light performance. After that the 'real' camera got little use, mostly taken on walks and holidays for wildlife shots and shots of the kids playing from a distance (real zoom).
For the subsequent two, the cameras were reasonable upgrades on the new phones. I'm not sure whether we'll bother with an upgrade for number 4 though, probably keep the phones until end of updates, which will be about another year. Cameras now are 'good enough' really.
(We've tended to buy mid-range phones but chosen primarily for cameras)
Interesting. I use my phone primarily as a phone. With WhatsApp a distant second. And Strava for running. I buy the cheapest with decent battery life. I replace it either when the screen is so cracked that it starts to impair use, or the battery stops recharging properly and no longer goes a full day between charges. Every 3-4 years?
Not interested in the quality of the camera - anything will do for a quick selfie. If I'm taking photos I take the proper camera. Instead of a camera on the phone, I'd rather have a phone on the camera, then I could leave the phone at home when I'm out for the day.
ICLR submission (not mine) flagged for Ethics Review based on the use of the term "black box." Reminiscent of the drama involving the term "Byzantine," though there is relevant guidance from ACM this time.
There are some crackers, but LOL...quantum supremacy....that's a no no too....The term has long been associated with crimes against humanity and genocide.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
No - there are even detailed marketing analysis of this kind of thing. There is a whole class of users who wait x "generations" before upgrading. Where X is a surprisingly constant number, I believe.
Our refresh rate was once per child for a time - got new phones before each of our first three kids. The first (2017) being the first phones we got that really had a serviceable camera. Still took my real camera in to the hospital (a now quite old, but decent at the time, compact). I remember being surprised afterwards that the phone pictures taken inside were better - better lowish light performance. After that the 'real' camera got little use, mostly taken on walks and holidays for wildlife shots and shots of the kids playing from a distance (real zoom).
For the subsequent two, the cameras were reasonable upgrades on the new phones. I'm not sure whether we'll bother with an upgrade for number 4 though, probably keep the phones until end of updates, which will be about another year. Cameras now are 'good enough' really.
(We've tended to buy mid-range phones but chosen primarily for cameras)
Interesting. I use my phone primarily as a phone. With WhatsApp a distant second. And Strava for running. I buy the cheapest with decent battery life. I replace it either when the screen is so cracked that it starts to impair use, or the battery stops recharging properly and no longer goes a full day between charges. Every 3-4 years?
Not interested in the quality of the camera - anything will do for a quick selfie. If I'm taking photos I take the proper camera. Instead of a camera on the phone, I'd rather have a phone on the camera, then I could leave the phone at home when I'm out for the day.
I'm similar. I do like a half-decent camera - I was sold my current phone on the basis it has three settings of optical zoom - but it's not a deal breaker: cheap is the main thing. The best thing about a phone camera is always having a camera on you, rather than having a brilliant camera.
If I could make one request of phone manufacturers it would be to manufacture a phone whose charging slot doesn't fill up with fluff. That is the biggest problem with every single phone I've owned.
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits).
And this is why NVidia has been holding most their mid-range graphics cards to 8GB for years, they want LLMs running remotely on their expensive datacentre chips, not a Geforce RTX in the box under your desk.
At the time the 4000 series was launched LLM was not the market it currently is.
You points may or may not be valid but only when we start to see the 5000 series GPUs arrive will we know whether your argument is valid
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
*pretends to understand this*
*nods knowledgeably*
*hastily feeds it into GPT for a translation*
Apologies!
Llama-3.1 is the model - from Meta (Facebook). 8B is the size of the model - 8 Billion weights. Instruct means it’s been instruction-tuned after the initial training, so it interacts like ChatGPT does.
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits). You trade some quality of output for a smaller memory footprint, because you inevitably throw away data when you do this.
KV cache is the Key-value cache - basically you store some of the current state of the LLM in the middle of an interaction so you can pick up where you left off without having to completely reprocess the entire conversation every time.
Your GPU still needs to do everything else you needed it to do before in order to see anything at all on the screen, so you reserve some memory for copies of the screen, application windows etc.
ICLR submission (not mine) flagged for Ethics Review based on the use of the term "black box." Reminiscent of the drama involving the term "Byzantine," though there is relevant guidance from ACM this time.
There are some crackers, but LOL...quantum supremacy....that's a no no too....The term has long been associated with crimes against humanity and genocide.
FFS. There’s no good / bad with black box / white box.
Although arguably the terms are somewhat opaque. Closed / Open or something equivalent would probably have been better.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
Apple has always been stingy with RAM. They don’t even list it in the official technical specs for their phones: https://www.apple.com/uk/iphone-16/specs/ & that’s because everyone else has more.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
12GB for an LLM? That will be fairly anaemic, won't it?
They’re a lot better than they have any right to be at that size frankly. And if you’re offline with no other sources, they might be enough to do the job.
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Which one?
Llama-3.1-8B-Instruct IIRC. The Q6_K quant is 6.2GB, so fits into a 8GB GPU with just enough space for a small KV cache & your display memory etc.
I really wasn’t expecting it to write working Haskell frankly.
*pretends to understand this*
*nods knowledgeably*
*hastily feeds it into GPT for a translation*
Apologies!
Llama-3.1 is the model - from Meta (Facebook). 8B is the size of the model - 8 Billion weights. Instruct means it’s been instruction-tuned after the initial training, so it interacts like ChatGPT does.
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits). You trade some quality of output for a smaller memory footprint, because you inevitably throw away data when you do this.
KV cache is the Key-value cache - basically you store some of the current state of the LLM in the middle of an interaction so you can pick up where you left off without having to completely reprocess the entire conversation every time.
Your GPU still needs to do everything else you needed it to do before in order to see anything at all on the screen, so you reserve some memory for copies of the screen, application windows etc.
I still don’t see why you’d need the LLM ON THE PHONE
Surely it can all be done remotely as now with GPT/Claude. I mean, I access Google incessantly (or I did until GPT/claude) I don’t have all of Google on my phone
The idea would be the phone would be operated remotely but with some basic backup in the device - like an emergency generator - if the power/signal goes down
However I do see there might be privacy issues? But then I think human privacy is coming to an end. Soon everyone will know everything about everyone everywhere
I agree! You don’t need the full LLM on the phone.
But if it’s not on the phone, you need an always on, fast internet connection to make the experience seamless, which isn’t universally available.
Given that these models are extremely expensive to run right now - the chips are very expensive (NVidia is making 80% margins, holy cow) and the power / cooling requirements are also steep - it’s not surprising that Apple are cautious.
It’s also possible that Apple’s models are just not good enough yet because they haven’t been willing to blatantly circumvent copyright laws in the way some of the AI specialist companies seem to have done.
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits).
And this is why NVidia has been holding most their mid-range graphics cards to 8GB for years, they want LLMs running remotely on their expensive datacentre chips, not a Geforce RTX in the box under your desk.
At the time the 4000 series was launched LLM was not the market it currently is.
You points may or may not be valid but only when we start to see the 5000 series GPUs arrive will we know whether your argument is valid
Yes the 5070 is going to have 16GB VRAM this time.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
No - there are even detailed marketing analysis of this kind of thing. There is a whole class of users who wait x "generations" before upgrading. Where X is a surprisingly constant number, I believe.
Our refresh rate was once per child for a time - got new phones before each of our first three kids. The first (2017) being the first phones we got that really had a serviceable camera. Still took my real camera in to the hospital (a now quite old, but decent at the time, compact). I remember being surprised afterwards that the phone pictures taken inside were better - better lowish light performance. After that the 'real' camera got little use, mostly taken on walks and holidays for wildlife shots and shots of the kids playing from a distance (real zoom).
For the subsequent two, the cameras were reasonable upgrades on the new phones. I'm not sure whether we'll bother with an upgrade for number 4 though, probably keep the phones until end of updates, which will be about another year. Cameras now are 'good enough' really.
(We've tended to buy mid-range phones but chosen primarily for cameras)
Interesting. I use my phone primarily as a phone. With WhatsApp a distant second. And Strava for running. I buy the cheapest with decent battery life. I replace it either when the screen is so cracked that it starts to impair use, or the battery stops recharging properly and no longer goes a full day between charges. Every 3-4 years?
Not interested in the quality of the camera - anything will do for a quick selfie. If I'm taking photos I take the proper camera. Instead of a camera on the phone, I'd rather have a phone on the camera, then I could leave the phone at home when I'm out for the day.
I'm similar. I do like a half-decent camera - I was sold my current phone on the basis it has three settings of optical zoom - but it's not a deal breaker: cheap is the main thing. The best thing about a phone camera is always having a camera on you, rather than having a brilliant camera.
If I could make one request of phone manufacturers it would be to manufacture a phone whose charging slot doesn't fill up with fluff. That is the biggest problem with every single phone I've owned.
They say the best camera is the one you have with you.
ICLR submission (not mine) flagged for Ethics Review based on the use of the term "black box." Reminiscent of the drama involving the term "Byzantine," though there is relevant guidance from ACM this time.
There are some crackers, but LOL...quantum supremacy....that's a no no too....The term has long been associated with crimes against humanity and genocide.
Anyway, good morning everyone. No snow in South Manchester but weirdly icy in a way I've only ever seen once before: the temperature must have dropped quite suddenly after yesterday evening's heavy sleet, leaving all the pavements and many of the roads covered by a solid and unbroken film of ice. It would have been hard to make them icier had it been done deliberately. Not particularly conducive to getting anywhere but quite pretty, with everywhere looking very shiny.
This happened in Oxford a few years ago right in the middle of the evening rush hour. Was absolute chaos.
The police had to close the A34 eventually & clear all the vehicles off it.
Freezing rain is much more common in continental climates than the UK, but it does occur. Usually with entrenched cold air underneath and a warm front moving in. We had a notorious one in the 90's in Wiltshire - predicted as heavy snow but the air moving in was too warm so it was freezing rain. Formed about a cm of solid ice on all surfaces. I ended up crawling to my house up the slope.
Even during the beast from the east we saw a period of freezing rain during the 18 hour snow storm. It led to a laying of crunchy ice about 2 inches down in the snow (as there was 6 inches of snow under it and 2 on top). A warm sector had moved over for a while during the night.
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
No - there are even detailed marketing analysis of this kind of thing. There is a whole class of users who wait x "generations" before upgrading. Where X is a surprisingly constant number, I believe.
Our refresh rate was once per child for a time - got new phones before each of our first three kids. The first (2017) being the first phones we got that really had a serviceable camera. Still took my real camera in to the hospital (a now quite old, but decent at the time, compact). I remember being surprised afterwards that the phone pictures taken inside were better - better lowish light performance. After that the 'real' camera got little use, mostly taken on walks and holidays for wildlife shots and shots of the kids playing from a distance (real zoom).
For the subsequent two, the cameras were reasonable upgrades on the new phones. I'm not sure whether we'll bother with an upgrade for number 4 though, probably keep the phones until end of updates, which will be about another year. Cameras now are 'good enough' really.
(We've tended to buy mid-range phones but chosen primarily for cameras)
Interesting. I use my phone primarily as a phone. With WhatsApp a distant second. And Strava for running. I buy the cheapest with decent battery life. I replace it either when the screen is so cracked that it starts to impair use, or the battery stops recharging properly and no longer goes a full day between charges. Every 3-4 years?
Not interested in the quality of the camera - anything will do for a quick selfie. If I'm taking photos I take the proper camera. Instead of a camera on the phone, I'd rather have a phone on the camera, then I could leave the phone at home when I'm out for the day.
People are *not* happy with the latest changes Strava are making...
Can we talk about the absolute mediocrity of the Apple 16 pro max. It’s basically identical to my 13 pro max. Nothing has changed in 3 years. And the big new innovation - “smart Siri, like ChatGPT” - has now apparently been delayed until… 2026
Apple have totally fucked this up. Go short Apple, they’re heading for trouble
Yes there's been no innovation for three generations at least with the iPhone, though I'm told the 17 is going to be the big one rather than just iterative.
But the 16 was meant to be the big one. It’s just a big disappointment. A nice phone but… meh
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
Battery life is the killer. LLMs need memory & great gobs of it, on top of the CPU required to run them & the combination needs a lot of power. Apple has always skimped on memory on their phones, in the interests of holding costs down & preserving battery life.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
Hold on, Apple phones cost a relative fortune. They are #2 by volume behind Samsung but #1 for revenue. Surely they should have plenty of memory ?
You can get an iPhone with 1Tb of memory. But that is the max - few people buy one of those. The processing would still be the killer.
Am I typical in buying a new phone every five or six years? The iphone 14 is a huge improvement over the old 7 but annual increments are comparatively underwhelming. The 7 still sits on my handlebars with enough charge to guide me on a three-hour ride in terra incognita. I wouldn't have enough breath to converse with Siri.
No - there are even detailed marketing analysis of this kind of thing. There is a whole class of users who wait x "generations" before upgrading. Where X is a surprisingly constant number, I believe.
Our refresh rate was once per child for a time - got new phones before each of our first three kids. The first (2017) being the first phones we got that really had a serviceable camera. Still took my real camera in to the hospital (a now quite old, but decent at the time, compact). I remember being surprised afterwards that the phone pictures taken inside were better - better lowish light performance. After that the 'real' camera got little use, mostly taken on walks and holidays for wildlife shots and shots of the kids playing from a distance (real zoom).
For the subsequent two, the cameras were reasonable upgrades on the new phones. I'm not sure whether we'll bother with an upgrade for number 4 though, probably keep the phones until end of updates, which will be about another year. Cameras now are 'good enough' really.
(We've tended to buy mid-range phones but chosen primarily for cameras)
Interesting. I use my phone primarily as a phone. With WhatsApp a distant second. And Strava for running. I buy the cheapest with decent battery life. I replace it either when the screen is so cracked that it starts to impair use, or the battery stops recharging properly and no longer goes a full day between charges. Every 3-4 years?
Not interested in the quality of the camera - anything will do for a quick selfie. If I'm taking photos I take the proper camera. Instead of a camera on the phone, I'd rather have a phone on the camera, then I could leave the phone at home when I'm out for the day.
I'm similar. I do like a half-decent camera - I was sold my current phone on the basis it has three settings of optical zoom - but it's not a deal breaker: cheap is the main thing. The best thing about a phone camera is always having a camera on you, rather than having a brilliant camera.
If I could make one request of phone manufacturers it would be to manufacture a phone whose charging slot doesn't fill up with fluff. That is the biggest problem with every single phone I've owned.
Agree with both, of course. But with young kids most of my pictures are snapshots of them. If out and about with potential for interesting photos beyond snapshots, then it's the real camera that I use. But that first 'decent' camera phone was when I started using a phone to actually take photos that I intended to keep and maybe even print.
Comments
Thing is there clearly are major problems with Apple Intelligence. It was meant to be ready to go with the new Apple 16. It isn’t. It’s risible. Then they said “yeah sorry wait til an update in the next few weeks”. Now it’s… 2026??
Meanwhile they’re clearly aware of the pressure coz they’ve announced a hook-up with ChatGPT in 2025 to replace Siri with Altman’s tech but it’s very murky and OpenAI don’t sound that keen
But it’s not just Apple in trouble here. Amazon are apparently having massive problems updating Alexa so it can match Claude, GPT4o etc
My best guess is that two or three companies (OpenAI, Anthropic, maybe the Chinese) HAVE mastered the tech and it’s very hard to copy and they have a big or crucial lead and they’re loathe to hand it over
OpenAI or Anthropic should make their own phone - that would be cat::pigeons
And it is being rumoured. Eg OpenAI are working quietly with Jonny Ive
https://www.nytimes.com/2024/09/21/technology/jony-ive-apple-lovefrom.html?smid=nytcore-ios-share&referringSource=articleShare
15-inch MacBook Pro – Battery Overheating/Fire Risk
Apple Ultracompact USB Power Adapter – Electrical Shock Risk
I know you love Apple, but you really do have blinkers on.
And for the record: Apple make good products. They market well. What they've done with the ARM instruction set is amazing. I can complement them when they do things well. But the fanbois who denigrate everyone else, and pretend Apple is better than it really is are really amusing.
We were told that 'the adults are now in charge'.
In reality 'the adults' did no proper planning and paid no attention to any detail.
Reeves seems to be an emptiness surrounded by a PPE education.
Remember how she banged on about being the first female chancellor ?
She's someone who believes she's achieved something by existing rather than by what she does.
https://www.youtube.com/watch?v=9A05Ggs5sME
Reported rumour that in the past when the UK PM visited the Dalai Lama, the word wet out across China that investment should not be made in the UK. And it basically wasn't. That sounds credible.
Also talk of whether "always mentioning human rights" as part of the talking points China checklist has made the faintest bit of difference in 30 years.
And potential aspect of the Le Pen trial, which I had not spotted, on the next French election.
They've even got bookmarks:
01:11 The farmers protest
10:50 Is the international order collapsing around us?
13:11 Violence in Amsterdam
16:25 Is Keir Starmer spending too much time abroad?
17:54 UK-China relations
18:45 What is the point of COP?
20:35 EU trade dwarfs every other trading relationship
22:30 Does the West really care about human rights?
26:18 Will Marine Le Pen be barred from standing?
29:49 What have Alastair and Rory been watching recently?
33:59 How did Jeffrey Epstein manage to protect himself?
All the current LLM companies are being heavily subsidised by investors in one way or another. Microsoft paid for OpenAI’s NVidia GPU farm, Amazon & Google have poured $billions into Anthropic.
You can sell LLMs to companies who can see the value in assisting or replacing parts of their workforce, but it’s entirely plausible that consumers don’t see enough benefit from LLMs personally in their daily lives & aren’t willing to pay enough to make the $ pencil out for Apple.
It's not really mocking woke though. Just mocking terrible adverts (like, for example, all perfume adverts - but also like the Jaguar advert, which my weathervane wife brought up unbidden this morning, in disbelief at the awfulness of it - she said she'd assumed all the way through it was some sort of parody (like this Bud advert quickly turns out to be), and was baffled when it turned out to be sincere.)
The next innovation has to be incorporating ChatGPT/claude level tech into a phone. Imagine a phone that could sync with/read your smartwatch and then chat with you - naturally - about the health data. “Hey max I’m a bit worried about your stress readings and heart rate. Maybe we could try this or this..”
Your phone would also talk to you naturally about your favourite sports, your marital problems, the future of Greenland, while admiring your photos of your kids, advising you on investments and booking those tickets to NYC
All this tech is basically available now. Put it in a phone: wow. Our phones will become like the Daimons in the philip Pullman books
Whichever company gets there first will make $$$$$
That's your classic NU10K thinking - I am a Proper Person by virtue of the eminence I have risen to. The key to the executive washroom is Mon Droit.
A voice-driven UX that only works when you have a 5G connection is going to be a terrible experience.
Rory, on the other hand, strikes me as perhaps a man with too much nuance: on the one hand, on the other hand, you're probably right Alastair.
Am I missing out?
Enter the great economist and her four months of trying to think what to do and kite flying possible tax rises in the media.
Followed by choosing the tax rise most likely to damage the economy.
Election Maps UK
@ElectionMapsUK
Litherland (Sefton) Council By-Election Result:
🌹 LAB: 43.7% (-30.3)
🌍 GRN: 34.0% (+27.1)
➡️ RFM: 12.4% (New)
🌳 CON: 4.2% (-0.6)
⚙️ WPB: 3.5% (New)
🧑🔧 TUSC: 2.2% (New)
No IND (-12.1) or LDM (-2.3) as previous.
Labour HOLD.
Changes w/ 2024.
https://enormo-haddock.blogspot.com/2024/11/las-vegas-pre-qualifying-2024.html
Looks fairly close.
ChatGPT/Claude level tech requires somewhere around 400Gb of weights I believe. You’re not getting that on a phone in the near future.
The police had to close the A34 eventually & clear all the vehicles off it.
I was expecting something more in your face like the Daniel Craig vodka ad
And don’t some android phones now have Gemini voice AI incorporated?
(I’ve no idea, I’ve only ever had iPhones for a decade… til now)
It will be interesting to see if Apple do integrate ChatGPT into their phones as rumoured, in 2025
https://www.tomsguide.com/ai/chatgpt-is-coming-to-iphone-ipad-and-mac-what-you-need-to-know
Sadly, in the case of Apple, they keep costs down by using their strength to break IP and steal tech. That's the major reason I dislike hem.
Brown was landed with a party which had been in power for well over 10 years and the infighting was getting under way.
At least that how it seemed to this sympathetic observer.
And Good Morning bro one and all; no snow in my garden although there are apparently a few sprinkles locally.
You could have argued (in the past) that iOS was better optimised & didn’t need as much RAM so it was a meaningless comparison that didn’t actually affect end-user experience. But in the days of LLMS, that argument doesn’t work any more: More is pretty much always better when it comes to LLM size & below 8GB or so the quality drops off dramatically.
Given that you need a few GB for the OS to run in, that means you need 12GB to run a local LLM on your phone. Apple put 8GB in the iPhone 16 & even that was a step up - the iPhone 15 had 6GB. Apple is not going to be running local LLMs until the iPhone 17 releases.
I spent so much time in the GPU compute space with strategises for loading to and from the various levels of memory.
Are you sure you don't mean 1 TB of storage capacity ?
"There is none, capacity is managed by the speed at which ChatGPT responds to drunken conversations"
In a world with always on high bandwidth internet, it does seem that we shouldn’t need on-device AI. Maybe Apple thinks that a service that’s only available some of the time isn’t good enough?
I’ve used an 8Gb LLM to write small programs for me. It did a lot better than I expected.
Obviously you don’t have to have it loaded if you’re not using it, but if you want to use it, it all has to be in RAM.
(You could try paging some of the individual matrices in and out as required, but given that LLMs are memory bandwidth limited you’ve just cut your token rate to that permitted by the bandwidth of your backend storage which is pitiful compared to the LPDDR memory chips.)
Why do you need it locally on the phone? You don’t
Just have some rudimentary back-up for the rare occasions you’re out of signal
I reckon OpenAI and Anthropic are playing hardball. They have the best tech after all
Given Microsoft’s strong relationship with Altman maybe they’re planning to launch the first true AI smartphone
I really wasn’t expecting it to write working Haskell frankly.
As Director of Public Prosecutions, I saw the terrible effect of anti-social behaviour first hand - and fought for justice for victims.
As Prime Minister, I am delivering on my promise to tackle it with new Respect Orders, so people face the consequences of their actions.
*nods knowledgeably*
*hastily feeds it into GPT for a translation*
There’s some evidence that OpenAI did something similar with their large models, but they’re generally fairly tight-lipped about the details.
The ads are supposed to be generally good though. Makes sense, I guess, the more unremarkable the product, the better the marketing you need to shift it! See*, e.g., supermarket Christmas ads.
*again, I haven't - yet, for this year at least - the one place I tend to actually see/watch ads is at the cinema and my most recent trip was just before the supermarket ads were unleased, I think - if they do show those in cinemas
“It seems your friend is quite knowledgeable about AI and smartphones! Let's break down what they said:
* Llama-3.1-8B-Instruct: This likely refers to a specific version of a large language model (LLM) called "Llama." LLMs are a type of artificial intelligence that can understand and generate human-like text.
* "3.1" likely signifies the major and minor version number of the model.
* "8B" probably indicates that this model has 8 billion parameters. Parameters are the values the model learns during training, and more parameters generally mean a more capable model.
* "Instruct" suggests this version of Llama is specifically fine-tuned to follow instructions and generate helpful responses.
* Q6_K quant: This refers to a technique called quantization, which reduces the size of the LLM for faster processing and lower memory requirements. "Q6_K" likely indicates a specific quantization method where the model's weights are compressed to 6 bits.
* 6.2GB: This is the size of the quantized LLM after applying the Q6_K technique.
* 8GB GPU: This refers to a graphics processing unit (GPU) with 8 gigabytes of memory. GPUs are specialized electronic circuits that accelerate the creation of images, videos, and other visual content. They are also very efficient at running the calculations needed for AI tasks like running LLMs.
* KV cache: This is a type of memory used to store frequently accessed data in an LLM, which can speed up processing.
* Haskell: This is a purely functional programming language known for its strong emphasis on immutability and its use in academia and industry.
In simpler terms: Your friend is impressed that a relatively small LLM (compressed to 6.2GB) can run on a smartphone with an 8GB GPU and even generate working code in a complex language like Haskell. This suggests that powerful AI capabilities could potentially be integrated into future smartphones.
The bigger picture: The rapid advancement of AI technology, combined with techniques like quantization, is making it increasingly feasible to run sophisticated AI models on devices with limited resources like smartphones. This opens up exciting possibilities for the future of mobile technology, where our phones could become truly intelligent assistants capable of understanding and responding to our needs in a much deeper way.”
Now she’s having a cup of tea
The GPU (Graphical Processing Unit) is the chip/board that used to be used for video games. It consists of lots and lots of fairly rubbish processors. Individually they are slow, but they can do stuff at the same time. And there are 10s of thousands of them. Which is great for computer graphics, some maths problems, and it turns out, neural networks (so called "AI").
So the Llama-3.1-8B-Instruct LLM will apparently fit on a fairly normal home computer and run acceptably.
Haskell is a classic, and rather interesting programming language.
Llama-3.1 is the model - from Meta (Facebook). 8B is the size of the model - 8 Billion weights. Instruct means it’s been instruction-tuned after the initial training, so it interacts like ChatGPT does.
Even an 8Billion weight model is too large to fit in an 8Gb GPU - the weights will be 32-bit floats, so the values of the weights are squeezed into a smaller space (fewer bits). You trade some quality of output for a smaller memory footprint, because you inevitably throw away data when you do this.
KV cache is the Key-value cache - basically you store some of the current state of the LLM in the middle of an interaction so you can pick up where you left off without having to completely reprocess the entire conversation every time.
Your GPU still needs to do everything else you needed it to do before in order to see anything at all on the screen, so you reserve some memory for copies of the screen, application windows etc.
Some have suggested that "Fit and proper" means "Anyone rich enough"
Since Sir Kier is such a football fan, I think it is relevant.
For the subsequent two, the cameras were reasonable upgrades on the new phones. I'm not sure whether we'll bother with an upgrade for number 4 though, probably keep the phones until end of updates, which will be about another year. Cameras now are 'good enough' really.
(We've tended to buy mid-range phones but chosen primarily for cameras)
🍿🍿🍿
"Russia just sent another ICBM from Astrakhan but looks to have malfunctioned, crashed somewhere in south central Ukraine, no longer on radar."
https://x.com/JayinKyiv/status/1859588318314295407
Surely it can all be done remotely as now with GPT/Claude. I mean, I access Google incessantly (or I did until GPT/claude) I don’t have all of Google on my phone
The idea would be the phone would be operated remotely but with some basic backup in the device - like an emergency generator - if the power/signal goes down
However I do see there might be privacy issues? But
then I think human privacy is coming to an end. Soon everyone will know everything about everyone everywhere
Not interested in the quality of the camera - anything will do for a quick selfie. If I'm taking photos I take the proper camera. Instead of a camera on the phone, I'd rather have a phone on the camera, then I could leave the phone at home when I'm out for the day.
ICLR submission (not mine) flagged for Ethics Review based on the use of the term "black box." Reminiscent of the drama involving the term "Byzantine," though there is relevant guidance from ACM this time.
https://x.com/thegautamkamath/status/1859787202173206840
Due to this,
https://www.acm.org/diversity-inclusion/words-matter
There are some crackers, but LOL...quantum supremacy....that's a no no too....The term has long been associated with crimes against humanity and genocide.
If I could make one request of phone manufacturers it would be to manufacture a phone whose charging slot doesn't fill up with fluff. That is the biggest problem with every single phone I've owned.
You points may or may not be valid but only when we start to see the 5000 series GPUs arrive will we know whether your argument is valid
https://huggingface.co/docs/transformers/model_doc/llama3
Although arguably the terms are somewhat opaque. Closed / Open or something equivalent would probably have been better. I agree! You don’t need the full LLM on the phone.
But if it’s not on the phone, you need an always on, fast internet connection to make the experience seamless, which isn’t universally available.
Given that these models are extremely expensive to run right now - the chips are very expensive (NVidia is making 80% margins, holy cow) and the power / cooling requirements are also steep - it’s not surprising that Apple are cautious.
It’s also possible that Apple’s models are just not good enough yet because they haven’t been willing to blatantly circumvent copyright laws in the way some of the AI specialist companies seem to have done.
Even during the beast from the east we saw a period of freezing rain during the 18 hour snow storm. It led to a laying of crunchy ice about 2 inches down in the snow (as there was 6 inches of snow under it and 2 on top). A warm sector had moved over for a while during the night.